Re: [foreman-dev] Problems with Katello Bastion UI when changing Foreman base URL

[gerrit . schwerthelm]

Hello Ewoud,

thanks for your informative and quick answer. As it looks to me that the 
required changes are touching quite a lot of sensitive points (from which 
many are not yet identified?), I think it's better for me not to continue 
with my undertaking and not to touch the sources.

For now, I will the just redirect ^/$ with httpd to point to an alias of 
where my own service sits. This way, only the dashboard of Foreman cannot 
be reached anymore, but the rest of Foreman and Katello is still intact and 
seems to work for everything I need.

I also already tried running Foreman in production on non-default ports, 
but it seems like this is even causing more issues as there seem to be some 
hard coded ports for the communication with Pulp in the sources (like 
https://github.com/Katello/katello/blob/6aca54157e579d6312a9e4a49609df0ffa1685b1/app/services/katello/proxy_status/pulp.rb#L45).
 

If you have any other ideas I could try out, I would be very grateful if 
you share them with me.

Many thanks and regards,

--Gerrit   


On Tuesday, September 12, 2017 at 12:29:05 PM UTC+2, Ewoud Kohl van 
Wijngaarden wrote:
>
> On Tue, Sep 12, 2017 at 02:56:06AM -0700, gerrit.sc...@avid.com 
>  wrote: 
> >I am having a setup where I use Foreman as a complete backend service. 
> >Another service works on the document root of the HTTPD server. In order 
> to 
> >have all pages of Foreman still reachable, I tried altering the base URL 
> of 
> >Foreman by changing the value for the foreman_url in the 
> foreman-installer. 
> >This is working great so far, all pages of Foreman now changed to 
> >https:///foreman, all API calls are working fine. 
> > 
> >However, when I try to visit a Katello page like content views or 
> products, 
> >the page cannot load and it ends up in an endless loop. I can provide 
> more 
> >detailed logging output if required, but as far as I can see, there is 
> not 
> >much information that can be found in the logs. 
> > 
> >Did anyone ever try changing the base URL and ran into this issue? Is 
> there 
> >a possibility to configure Katello such that the pages work with an 
> altered 
> >base? Is there maybe a setting in the foreman-installer that I need to 
> >change to make it work? 
> > 
> >Thanks for reading and all the best, 
>
> https://github.com/Katello/puppet-katello/pull/211 was merged just last 
> week so I'm guessing few people do. There may be other dragons hiding 
> because deployment_url is considered relative in post_sync_url now but 
> in candlepin it might have been considered absolute so I dug in deeper 
> (as I should have when I merged 211) and it looks like the whole 
> deployment_url is unused in puppet-candlepin and we should just remove 
> that part. https://github.com/Katello/puppet-candlepin/pull/82 together 
> with https://github.com/Katello/puppet-katello/pull/214 should take care 
> of that. 
>
> Looking through the source then I see some examples that should work but 
> many that do not. What should work is using the foreman_url helper like 
>
> https://github.com/Katello/katello/blob/b02526bea7026560b2d6d66fac9038cfeb74bab9/app/assets/javascripts/katello/hosts/activation_key_edit.js#L22
>  
>
> What doesn't work is plain URLs like 
>
> https://github.com/Katello/katello/blob/b02526bea7026560b2d6d66fac9038cfeb74bab9/app/assets/javascripts/katello/hosts/host_and_hostgroup_edit.js#L28
>  
>
> In short: I think you're entering uncharted/unsupported territory. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] Problems with Katello Bastion UI when changing Foreman base URL

[gerrit . schwerthelm]

Hey all,

I am having a setup where I use Foreman as a complete backend service. 
Another service works on the document root of the HTTPD server. In order to 
have all pages of Foreman still reachable, I tried altering the base URL of 
Foreman by changing the value for the foreman_url in the foreman-installer. 
This is working great so far, all pages of Foreman now changed to 
https:///foreman, all API calls are working fine. 

However, when I try to visit a Katello page like content views or products, 
the page cannot load and it ends up in an endless loop. I can provide more 
detailed logging output if required, but as far as I can see, there is not 
much information that can be found in the logs.

Did anyone ever try changing the base URL and ran into this issue? Is there 
a possibility to configure Katello such that the pages work with an altered 
base? Is there maybe a setting in the foreman-installer that I need to 
change to make it work?

Thanks for reading and all the best,

--Gerrit

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] Re: Using sync task for running Ansible plays hangs Foreman

[gerrit . schwerthelm]

It actually just seems like the entire API is blocked when a sync task is 
triggered directly in the controller method. Does it have something to do 
with concurrency in Rails? Maybe also resources get locked producing a 
deadlock? Still would be happy about some suggestions.
 

On Monday, March 13, 2017 at 11:09:45 AM UTC+1, gerrit.sc...@avid.com wrote:
>
> Hello,
>
> I have an issue with triggering synchronous tasks in foreman-tasks. I have 
> written a plugin that is triggering an Ansible Play as a foreman-tasks' 
> sync_task (the foreman_ansible plugin itself uses only asynchronous 
> tasks, but I want the request in my plugin to particularly block until the 
> triggered Ansible task has completed). During playbook execution, the 
> Ansible callback plugin reports gathered facts to the Foreman facts API, 
> which I need later. However, as soon as the callback plugin contacts the 
> facts API, Foreman starts hanging completely, not answering any requests 
> any longer. Restarting httpd makes Foreman in production usable again. 
> After restarting and trying the call again, the entire procedure often 
> works as expected, which makes it even more incomprehensible for me. The 
> problem does not happen at all when the Ansible callback plugin is not 
> configured. In this case the call always blocks and succeeds as expected. 
> Does anybody have any idea what could be the reason for this behaviour? And 
> is there any way to get around this problem? I'd be very grateful for any 
> hints.
>
> Regards
>
> --Gerrit
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Fetch continuous output of non-delegated foreman / dynflow tasks

[gerrit . schwerthelm]

Hi Ivan,

this is really great and working for me. Thanks! I did not know I can 
access the planned actions this way. 

The case is that I wrote a small Foreman plugin, which -- apart from other 
things -- offers a customized artifact upload. The artifact is an archive 
containing repository contents as well as a metadata file. The metadata 
file defines all the products and repositories. And what the plugin does is 
inflating the archive, creating all the products, repositories and 
uploading the contents to the repository automatically. The archive 
inflation for example is the first task of my minimal example. It's very 
valuable to be able to see console output about what is happening during 
this process.

Thanks very much again!

All the best,

--Gerrit


On Tuesday, March 21, 2017 at 9:46:58 PM UTC+1, Ivan Necas wrote:
>
> gerrit.sc...@avid.com  writes: 
>
> > Hi, 
> > 
> > I was studying the dynflow documentation, but there are some pieces or 
> > examples missing in order for me to understand how to do this and I 
> cannot 
> > find examples in the Katello sources either. Therefore, I would 
> appreciate 
> > some help. 
> > 
> > I have written one main task that is a composition of smaller tasks: 
> > 
> > class MainAction < Actions::EntryAction 
> >   def plan(args) 
> > sequence do 
> >   plan_action ::Actions::MyPlugin::FirstTask, args 
> >   plan_action ::Actions:: MyPlugin::SecondTask, args 
> > end 
> > plan_self 
> >   end 
> > 
> > ... 
> > 
> > I have wrapped the smaller tasks into runner objects derived from 
> > ForemanTasksCore::Runner::Base and use publish_data() to emit output for 
> > the console. In the dynflow console, I can observe that the tasks are 
> > publishing output when browsing into the smaller tasks. But what is 
> unclear 
> > for me is how I can fetch this output and show it in the console of the 
> > main action such that I can observe the progress in the foreman-task 
> task 
> > page. I believe it needs to work somehow by including 
> > ::Actions::Helpers::WithContinuousOutput? I can see something similar 
> > working with delegated actions in the foreman_ansible plugin. However, I 
> do 
> > not need delegated actions as everything happens locally anyway. 
> > 
> > Thanks for reading. Looking forward to an answer. 
>
> Hi Gerrit, 
>
> The easiest way you could go is, on the entry action of the task, to 
> override the humanized_output, as done in [1]. In there, just to pass 
> the output from a sub-action. 
>
> The simplest way to go with our example would be 
>
> ``` 
> def humanized_output 
>   planned_actions(::Actions::MyPlugin::FirstTask).first.output 
> end 
> ``` 
>
> The continious output [2] is there just as a interface to keep the 
> format of the output messages in sync between the task and the UI, when 
> relying on the output that it produces: this is the format we are using 
> for ansible and remote execution. 
>
> I'm interested into the case you're trying to solve to see, what would 
> be the best way to address it (+ I'm curious of course :) 
>
> [1] - 
> https://github.com/theforeman/foreman-tasks/blob/826b902cb8604e9ba1332fa5468f6ea49c7e980f/app/lib/actions/helpers/with_delegated_action.rb#L24
>  
> [2] - 
>
> https://github.com/theforeman/foreman-tasks/blob/826b902cb8604e9ba1332fa5468f6ea49c7e980f/lib/foreman_tasks_core/continuous_output.rb
>  
>
> -- Ivan 
>
> > 
> > Regards, 
> > 
> > --Gerrit 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "foreman-dev" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to foreman-dev...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] Fetch continuous output of non-delegated foreman / dynflow tasks

[gerrit . schwerthelm]

Hi,

I was studying the dynflow documentation, but there are some pieces or 
examples missing in order for me to understand how to do this and I cannot 
find examples in the Katello sources either. Therefore, I would appreciate 
some help.

I have written one main task that is a composition of smaller tasks:

class MainAction < Actions::EntryAction
  def plan(args)
sequence do
  plan_action ::Actions::MyPlugin::FirstTask, args
  plan_action ::Actions:: MyPlugin::SecondTask, args
end
plan_self
  end

...

I have wrapped the smaller tasks into runner objects derived from 
ForemanTasksCore::Runner::Base and use publish_data() to emit output for 
the console. In the dynflow console, I can observe that the tasks are 
publishing output when browsing into the smaller tasks. But what is unclear 
for me is how I can fetch this output and show it in the console of the 
main action such that I can observe the progress in the foreman-task task 
page. I believe it needs to work somehow by including 
::Actions::Helpers::WithContinuousOutput? I can see something similar 
working with delegated actions in the foreman_ansible plugin. However, I do 
not need delegated actions as everything happens locally anyway.

Thanks for reading. Looking forward to an answer.

Regards,

--Gerrit

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Using sync task for running Ansible plays hangs Foreman

[gerrit . schwerthelm]

Hey! Yes, I can. This is form the log:

11:30:56 rails.1   | 2017-03-13T11:30:56 522230d8 [app] [D] 

11:30:56 rails.1   | 2017-03-13T11:30:56 522230d8 [app] [D] 

11:30:56 rails.1   | 2017-03-13T11:30:56 522230d8 [app] [I] Started POST 
"/my_plugin/api/v2/discover_host" for 10.129.206.169 at 2017-03-13 11:30:56 
+0100

11:30:56 rails.1   | 2017-03-13T11:30:56 522230d8 [app] [I] Processing by 
Api::V2::DiscoveryController#discover_host as */*

11:30:56 rails.1   | 2017-03-13T11:30:56 522230d8 [app] [I]   Parameters: 
{"host"=>"[FILTERED]", "location_id"=>1, "organization_id"=>1, 
"password"=>"[FILTERED]", "apiv"=>"v2", "discovery"=>{"host"=>"[FILTERED]", 
"location_id"=>1, "organization_id"=>1, "password"=>"[FILTERED]"}}

11:30:56 rails.1   | 2017-03-13T11:30:56 522230d8 [app] [D] Setting current 
user thread-local variable to admin

11:30:56 rails.1   | 2017-03-13T11:30:56 522230d8 [foreman-tasks/dynflow] 
[D] ExecutionPlan 33714ed3-a05e-4eb6-b5d8-88b559e44d8c  pending >>  
planning

11:30:57 rails.1   | 2017-03-13T11:30:56 522230d8 [foreman-tasks/dynflow] 
[D]  Step 33714ed3-a05e-4eb6-b5d8-88b559e44d8c: 1   pending >>   
running in phase Plan Actions::ForemanAnsible::PlayHostRole

11:30:58 rails.1   | 2017-03-13T11:30:58 522230d8 [foreman-tasks/dynflow] 
[D]  Step 33714ed3-a05e-4eb6-b5d8-88b559e44d8c: 2   pending >>   
running in phase Plan ForemanAnsibleCore::Actions::RunPlaybook

11:30:58 rails.1   | 2017-03-13T11:30:58 522230d8 [foreman-tasks/dynflow] 
[D]  Step 33714ed3-a05e-4eb6-b5d8-88b559e44d8c: 2   running >>   
success in phase Plan ForemanAnsibleCore::Actions::RunPlaybook

11:30:58 rails.1   | 2017-03-13T11:30:58 522230d8 [foreman-tasks/dynflow] 
[D]  Step 33714ed3-a05e-4eb6-b5d8-88b559e44d8c: 1   running >>   
success in phase Plan Actions::ForemanAnsible::PlayHostRole

11:30:58 rails.1   | 2017-03-13T11:30:58 522230d8 [foreman-tasks/dynflow] 
[D] ExecutionPlan 33714ed3-a05e-4eb6-b5d8-88b559e44d8c planning >>   
planned

11:30:58 rails.1   | 2017-03-13T11:30:58 ffd3b366 [foreman-tasks/dynflow] 
[D] ExecutionPlan 33714ed3-a05e-4eb6-b5d8-88b559e44d8c  planned >>   
running

11:30:58 rails.1   | 2017-03-13T11:30:58 ffd3b366 [foreman-tasks/dynflow] 
[D]  Step 33714ed3-a05e-4eb6-b5d8-88b559e44d8c: 5   pending >>   
running in phase  Run Actions::ForemanAnsible::PlayHostRole

11:30:58 rails.1   | 2017-03-13T11:30:58 ffd3b366 [foreman-tasks/dynflow] 
[D]  Step 33714ed3-a05e-4eb6-b5d8-88b559e44d8c: 3   pending >>   
running in phase  Run ForemanAnsibleCore::Actions::RunPlaybook

11:30:58 rails.1   | 2017-03-13T11:30:58 ffd3b366 [foreman-tasks/dynflow] 
[D]  Step 33714ed3-a05e-4eb6-b5d8-88b559e44d8c: 5   running >> 
suspended in phase  Run Actions::ForemanAnsible::PlayHostRole

11:30:58 rails.1   | 2017-03-13T11:30:58 ffd3b366 [foreman-tasks/dynflow] 
[D]  Step 33714ed3-a05e-4eb6-b5d8-88b559e44d8c: 3   running >> 
suspended in phase  Run ForemanAnsibleCore::Actions::RunPlaybook

11:30:58 rails.1   | 2017-03-13T11:30:58 ffd3b366 [foreman-tasks/dynflow] 
[D] start runner fc611544-441c-412b-b396-58765309bca8

11:30:58 rails.1   | 2017-03-13T11:30:58 ffd3b366 [foreman-tasks/dynflow] 
[D] [foreman_ansible] - Initializing Ansible Runner

11:30:58 rails.1   | 2017-03-13T11:30:58 ffd3b366 [foreman-tasks/dynflow] 
[D] [foreman_ansible] - Running command 
[{"JSON_INVENTORY_FILE"=>"/tmp/d20170313-8939-1ecph8h/foreman-inventories/fc611544-441c-412b-b396-58765309bca8"},
 
"ansible-playbook", "-i", "/root/foreman_ansible/bin/json_inventory.sh", 
"/tmp/d20170313-8939-1ecph8h/foreman-playbook-fc611544-441c-412b-b396-58765309bca8.yml"]

11:30:58 rails.1   | 2017-03-13T11:30:58 ffd3b366 [foreman-tasks/dynflow] 
[D] refresh runner fc611544-441c-412b-b396-58765309bca8

11:30:58 rails.1   | 2017-03-13T11:30:58 ffd3b366 [foreman-tasks/dynflow] 
[D] refreshing runner

11:30:58 rails.1   | 2017-03-13T11:30:58 ffd3b366 [foreman-tasks/dynflow] 
[D] planning to refresh fc611544-441c-412b-b396-58765309bca8

11:31:00 rails.1   | 2017-03-13T11:31:00 ffd3b366 [foreman-tasks/dynflow] 
[D] refresh runner fc611544-441c-412b-b396-58765309bca8

11:31:00 rails.1   | 2017-03-13T11:31:00 ffd3b366 [foreman-tasks/dynflow] 
[D] refreshing runner

11:31:00 rails.1   | 2017-03-13T11:31:00 ffd3b366 [foreman-tasks/dynflow] 
[D] planning to refresh fc611544-441c-412b-b396-58765309bca8

11:31:00 rails.1   | 2017-03-13T11:31:00 ffd3b366 [foreman-tasks/dynflow] 
[D]  Step 33714ed3-a05e-4eb6-b5d8-88b559e44d8c: 3 got event 
#

11:31:00 rails.1   | 2017-03-13T11:31:00 ffd3b366 [foreman-tasks/dynflow] 
[D]  Step 33714ed3-a05e-4eb6-b5d8-88b559e44d8c: 3 suspended >>   
running in phase  Run ForemanAnsibleCore::Actions::RunPlaybook

11:31:00 rails.1   | 2017-03-13T11:31:00 ffd3b366 [foreman-tasks/dynflow] 
[D]  Step 33714ed3-a05e-4eb6-b5d8-88b559e44d8c: 3   running >> 
suspended in phase  Run ForemanAnsibleCore::Actions:

[foreman-dev] Using sync task for running Ansible plays hangs Foreman

[gerrit . schwerthelm]

Hello,

I have an issue with triggering synchronous tasks in foreman-tasks. I have 
written a plugin that is triggering an Ansible Play as a foreman-tasks' 
sync_task (the foreman_ansible plugin itself uses only asynchronous tasks, 
but I want the request in my plugin to particularly block until the 
triggered Ansible task has completed). During playbook execution, the 
Ansible callback plugin reports gathered facts to the Foreman facts API, 
which I need later. However, as soon as the callback plugin contacts the 
facts API, Foreman starts hanging completely, not answering any requests 
any longer. Restarting httpd makes Foreman in production usable again. 
After restarting and trying the call again, the entire procedure often 
works as expected, which makes it even more incomprehensible for me. The 
problem does not happen at all when the Ansible callback plugin is not 
configured. In this case the call always blocks and succeeds as expected. 
Does anybody have any idea what could be the reason for this behaviour? And 
is there any way to get around this problem? I'd be very grateful for any 
hints.

Regards

--Gerrit

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] [Hammer CLI] Hidden flag for all parameters

[gerrit . schwerthelm]

Hey all,

I saw that it is possible to hide values in the host parameters, host group 
parameters and global parameters. However, there is no flag provided in the 
Hammer CLI to create parameters as hidden values for these instances. Do I 
miss something or is there any reason why this is not possible? 

Thanks for reading and kind regards

--Gerrit

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] [Ansible] Support for playbook execution, not just role execution

[gerrit . schwerthelm]

Hey Daniel, 

thanks for your answer. 

Actually, I am also working with role dependencies at the moment, which is 
fine for the time being for structuring a play. The new plugin feature of 
running roles against hostgroups reduced the need for me to run playbooks 
directly. However, it's a bit of a pity that all the playbook-level 
features of Ansible like serial execution cannot be used at the moment. I 
suppose, we can leave it with just roles for now but maybe we need to find 
some ways in order to be able to make use of the playbook-level Ansible 
features if required. 

--Gerrit


On Friday, December 23, 2016 at 4:06:27 PM UTC+1, Daniel Lobato wrote:
>
> On 12/23, gerrit.sc...@avid.com  wrote: 
> > Hello, 
> > 
> > currently, the Ansible plugin is limited to execute Ansible roles. 
> However, 
> > there are several reasons for considering direct Ansible playbook 
> > execution. 
> > 
> > Just with roles, there is a lack of control over the execution order as 
> the 
> > structuring element, the playbook, is statically created by the plugin. 
> In 
> > addition to that, some fundamental features of Ansible on playbook-level 
> > cannot be used with roles, for example strategies 
> > <
> http://docs.ansible.com/ansible/playbooks_delegation.html#rolling-update-batch-size>
>  
>
>
> Dependencies and ordering with roles can be accomplished at the role 
> level (not via Foreman) - check 
> http://docs.ansible.com/ansible/playbooks_roles.html#role-dependencies 
>
> > Therefore, I think, that the plugin should support playbook execution, 
> not 
> > just role execution. 
> > 
> > What do you think about this? I would really appreciate some more 
> thoughts 
> > before starting to work on this. 
>
> I'm afraid it's an area where the project clashes with Ansible Tower 
> quite a lot. To be fair I think playbooks would fit better as another 
> provider for Foreman remote execution model. Unfortunately I couldn't 
> finish the provider, but it worked (for my dev usecase :) ) 
>
> https://github.com/theforeman/foreman_ansible/pull/18 
>
> > 
> > Thanks for reading. 
> > 
> > --Gerrit 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "foreman-dev" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to foreman-dev...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>
>
> -- 
> Daniel Lobato Garcia 
>
> @dLobatog 
> blog.daniellobato.me 
> daniellobato.me 
>
> GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30 
> Keybase: https://keybase.io/elobato 
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] [Ansible] Support for playbook execution, not just role execution

[gerrit . schwerthelm]

Hello,

currently, the Ansible plugin is limited to execute Ansible roles. However, 
there are several reasons for considering direct Ansible playbook 
execution. 

Just with roles, there is a lack of control over the execution order as the 
structuring element, the playbook, is statically created by the plugin. In 
addition to that, some fundamental features of Ansible on playbook-level 
cannot be used with roles, for example strategies 

. 

Therefore, I think, that the plugin should support playbook execution, not 
just role execution. 

What do you think about this? I would really appreciate some more thoughts 
before starting to work on this. 

Thanks for reading.

--Gerrit

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.