Thanks a lot for the clarifications Konstantinos! I opted to use the mahout interface and managed to overcome the previous issue.
Then I bumped into this bug: https://bugs.launchpad.net/juju-deployer/+bug/1575863 I followed the suggested workaround (mkdir ~/.juju) and now I get an "error getting env api endpoints". Any ideas? Panagiotis 2016-11-23 15:54:36 Starting deployment of lxd:admin/default 2016-11-23 15:54:36 Error getting env api endpoints, env bootstrapped? 2016-11-23 15:54:36 Command (juju api-endpoints -e lxd:admin/default) Output: 2016-11-23 15:54:36 Deployment stopped. run time: 0.15 E ====================================================================== ERROR: setUpClass (__main__.TestDeploy) ---------------------------------------------------------------------- Traceback (most recent call last): File "./tests/01-giraph-test.py", line 44, in setUpClass cls.d.setup(timeout=3600) File "/usr/lib/python3/dist-packages/amulet/deployer.py", line 704, in setup subprocess.check_call(shlex.split(cmd)) File "/usr/lib/python3.5/subprocess.py", line 581, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['juju-deployer', '-W', '-c', '/tmp/amulet-juju-deployer-o1zyf3xd/deployer-schema.json', '-e', 'lxd:admin/default', '-t', '3700', 'lxd:admin/default']' returned non-zero exit status 1 ---------------------------------------------------------------------- Ran 0 tests in 4.738s FAILED (errors=1) 2016-11-22 16:31 GMT+02:00 Konstantinos Tsakalozos <[email protected]>: > Cool! The error that you are getting now says that the giraph charm has no > giraph relation [0]. > > Looking at the metadata.yaml, you are using the mahout interface to relate > to the hadoop client. So the relation call should look like: > cls.d.relate('giraph:mahout', 'client:mahout') . This will work for now > since both Mahout and Giraph just want to add jars on the hadoop-client > charm. However, on the long run we should refactor charms such as > hadoop-client, spark, pig, etc, so that they have an interface exactly for > that purpose (adding jars to the file system). For now you have two options > either keep using the mahout interface or use the "juju-info" [1] interface > that is present in all charms. Note, however, that if you do use "juju-info" > your charm would be a subordinate to any other charm even if that does not > make sense. > > [0] > http://pythonhosted.org/amulet/amulet.html?highlight=relate#amulet.deployer.Deployment.relate > [1] https://jujucharms.com/docs/2.0/authors-implicit-relations > > On Tue, Nov 22, 2016 at 4:02 PM, Panagiotis Liakos <[email protected]> > wrote: >> >> Thanks Konstantinos, you were right. Now the script progresses a little >> further: >> >> $ python3 ./tests/01-giraph-test.py >> E >> ====================================================================== >> ERROR: setUpClass (__main__.TestDeploy) >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File "./tests/01-giraph-test.py", line 42, in setUpClass >> cls.d.relate('giraph:giraph', 'client:mahout') >> File "/usr/lib/python3/dist-packages/amulet/deployer.py", line 431, in >> relate >> raise ValueError('%s does not exist for %s' % (rel, srv)) >> ValueError: giraph does not exist for giraph >> >> ---------------------------------------------------------------------- >> Ran 0 tests in 2.285s >> >> FAILED (errors=1) >> >> >> Is this a namespace issue? Any ideas? >> >> Panagiotis >> >> 2016-11-22 14:16 GMT+02:00 Konstantinos Tsakalozos >> <[email protected]>: >> > Hi Panagiotis, >> > >> > The 404 error seems to be because the Giraph charm is not yet >> > promulgated >> > (your charm currently lives in cs:~panagiotisl/giraph). Could you try >> > the >> > same test only this time with a charm revision you have pushed under >> > your >> > namespace in line >> > >> > https://github.com/panagiotisl/bigtop/blob/master/bigtop-packages/src/charm/giraph/layer-giraph/tests/01-giraph-test.py#L29 >> > ? You can also use a local path pointing to where your charm build >> > output >> > is. >> > >> > Glad the multiple SLF4J bindings are not causing any problem. >> > >> > Thanks, >> > Konstantinos >> > >> > On Tue, Nov 22, 2016 at 1:15 PM, Panagiotis Liakos <[email protected]> >> > wrote: >> >> >> >> Dear Merlijn and Konstantinos, >> >> >> >> Thanks a lot for the very valuable information. I will have another >> >> look at the documentation for writing tests and debugging and I'll try >> >> to add some logs in my test. >> >> >> >> When I execute my smoke-test (through ssh to giraph) I get the >> >> following (which is exactly what I expect): >> >> https://pastebin.ubuntu.com/23516317/ >> >> >> >> When I execute the 01-giraph-test.py (and after I installed amulet) I >> >> get the following: https://pastebin.ubuntu.com/23516415/ >> >> >> >> The URL mentioned in the HTTPError exception is valid for mahout but >> >> not for giraph so I guess I am requesting something that is not there >> >> (yet). >> >> >> >> Thanks again, I'll keep you updated. >> >> >> >> Panagiotis >> >> >> >> >> >> 2016-11-22 12:08 GMT+02:00 Konstantinos Tsakalozos >> >> <[email protected]>: >> >> > Hi Panagiotis, >> >> > >> >> > Merlijn is right, the output of the smoke-test script is available in >> >> > the >> >> > juju debug logs (juju debug-log). Here is what I got when running >> >> > your >> >> > smoke-test: http://pastebin.ubuntu.com/23516167/ (multiple SLF4J >> >> > bindings) >> >> > It seems the slf4j-log4j12-1.7.5.jar jar from Giraph is slightly >> >> > older >> >> > that >> >> > the slf4j-log4j12-1.7.10.jar from Hadoop. >> >> > >> >> > In order to add some output to the smoke-test action you could call >> >> > "action >> >> > set" and/or "action fail" as we do here >> >> > >> >> > >> >> > https://github.com/panagiotisl/bigtop/blob/master/bigtop-packages/src/charm/hbase/layer-hbase/actions/smoke-test >> >> > >> >> > Thank you for your work, >> >> > Konstantinos >> >> > >> >> > >> >> > On Tue, Nov 22, 2016 at 11:06 AM, Merlijn Sebrechts >> >> > <[email protected]> wrote: >> >> >> >> >> >> Hi Panagiotis Liakos >> >> >> >> >> >> >> >> >> Cool that you're charming Giraph! Feel free to let me know when you >> >> >> have a >> >> >> working prototype, some of my colleagues are interested in using >> >> >> Giraph. >> >> >> >> >> >> Now for your question; tests and actions are two different things. >> >> >> >> >> >> Actions are similar to hooks; they run on the charm unit itself. The >> >> >> mahout smoke-test is an action. You run them using `run-action`, you >> >> >> can >> >> >> debug them using `juju debug-hooks` and all output gets logged by >> >> >> juju >> >> >> (you >> >> >> can see the output in `juju debug-log`). More info on actions: >> >> >> https://jujucharms.com/docs/stable/actions >> >> >> Tests are scripts that run on your own machine that deploy and test >> >> >> charms. You run them using bundletester. More info on tests: >> >> >> https://jujucharms.com/docs/2.0/developer-testing >> >> >> >> >> >> So if you want to alter the `smoke-test` action; you should alter >> >> >> this >> >> >> file: >> >> >> >> >> >> >> >> >> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/charm/mahout/layer-mahout/actions/smoke-test >> >> >> >> >> >> The actual tests of the Mahout Charm use the smoke-test action to >> >> >> verify >> >> >> the deployment. That is what you see at line 52: >> >> >> >> >> >> >> >> >> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/charm/mahout/layer-mahout/tests/01-mahout-test.py#L52 >> >> >> `self.mahout.run_action('smoke-test')` calls `juju run-action` in >> >> >> the >> >> >> background. >> >> >> >> >> >> Does that help you a bit? >> >> >> >> >> >> >> >> >> >> >> >> Kind regards >> >> >> Merlijn >> >> >> >> >> >> 2016-11-22 9:42 GMT+01:00 Panagiotis Liakos <[email protected]>: >> >> >>> >> >> >>> Hi all, >> >> >>> >> >> >>> I am trying to build a charm for Apache Giraph. I have used the >> >> >>> mahout >> >> >>> charm ( >> >> >>> >> >> >>> >> >> >>> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/charm/mahout >> >> >>> ) and made the necessary changes to deploy giraph instead of >> >> >>> mahout. >> >> >>> >> >> >>> Deployment seems to work fine and I am able to submit giraph jobs >> >> >>> through an ssh connection. >> >> >>> >> >> >>> Now I am trying to create a test similar to the smoke-test of the >> >> >>> mahout charm ( >> >> >>> >> >> >>> >> >> >>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/charm/mahout/layer-mahout/actions/smoke-test >> >> >>> ) >> >> >>> >> >> >>> I have successfully run the script that I have included in my >> >> >>> smoke-test through ssh and I >> >> >>> would expect that the test would also execute without errors. >> >> >>> However, I am not at all familiar with the 'run-action' process of >> >> >>> juju and I have no idea how I should alter the following file: >> >> >>> >> >> >>> >> >> >>> >> >> >>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/charm/mahout/layer-mahout/tests/01-mahout-test.py >> >> >>> >> >> >>> In particular, I am puzzled by the way the result status is set in >> >> >>> this example. Simply changing the references of mahout to giraph >> >> >>> does >> >> >>> not seem to work. You can see my file here: >> >> >>> >> >> >>> >> >> >>> >> >> >>> https://github.com/panagiotisl/bigtop/blob/master/bigtop-packages/src/charm/giraph/layer-giraph/tests/01-giraph-test.py >> >> >>> >> >> >>> When I execute show-action-output I receive very limited >> >> >>> information: >> >> >>> message: exit status 1 >> >> >>> status: failed >> >> >>> timing: >> >> >>> completed: 2016-11-21 15:48:01 +0000 UTC >> >> >>> enqueued: 2016-11-21 15:47:38 +0000 UTC >> >> >>> started: 2016-11-21 15:47:40 +0000 UTC >> >> >>> >> >> >>> Is there a way I can view the full output of the 'smoke-test' >> >> >>> execution? >> >> >>> >> >> >>> Thank you, >> >> >>> Panagiotis Liakos >> >> >>> >> >> >>> -- >> >> >>> Juju mailing list >> >> >>> [email protected] >> >> >>> Modify settings or unsubscribe at: >> >> >>> https://lists.ubuntu.com/mailman/listinfo/juju >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Juju mailing list >> >> >> [email protected] >> >> >> Modify settings or unsubscribe at: >> >> >> https://lists.ubuntu.com/mailman/listinfo/juju >> >> >> >> >> > >> > >> > > > -- Juju mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
