Re: pax-jdbc-dbcp2 and connection pooling

[dpravin]

Hi Achim,

Here is link to the pull request.

https://github.com/ops4j/org.ops4j.pax.jdbc/pull/29


Thanks,
Pravin



--
View this message in context: 
http://karaf.922171.n3.nabble.com/pax-jdbc-dbcp2-and-connection-pooling-tp4048791p4048851.html
Sent from the Karaf - User mailing list archive at Nabble.com.

Re: Configuration file handling?

[Jean-Baptiste Onofré]

You can use the append attribute on feature config.

However, it won't override, it's only append. And it's the expected 
behavior as if an user changes the properties, we don't want to override 
the "custom" values.


Regards
JB

On 11/28/2016 05:11 PM, Tim Ward wrote:

OK, nearly there, thanks all so far!

So  updates the etc folder, and keeps all my data in one place,
so that looks like the way to go ...

... give or take the upgrade problem. So what happens if

(a) a new version of a feature is installed which sets a new config item
which wasn't already in the etc file
(b) a new version of a feature is installed which sets a config item
which is already in the etc file with a different value

?

If the answers are

(a) the new config item gets written to the etc file
(b) the existing config item doesn't get overwritten

then that solves the problem, but I don't see that much detail in the
documentation?

Oh, and, I note that deleting the contents of the data directory is
stated to clear your Karaf down to a known state. But in fact it doesn't
do this, because etc is outside data, so garbage config files and items
can get left behind. What do people normally do about this?

On 28/11/2016 16:03, Jean-Baptiste Onofré wrote:

Actually, feature  now populate the etc folder as well (since
Karaf 4.0.5 AFAIR).

Regards
JB

On 11/28/2016 02:50 PM, Christian Schneider wrote:

You already found the confiFile option for features. This is the most
widely used option.
The alternative is the config option which simply adds the config in
config admin but not in etc.

Both variants do not cover the upgrade case. A simply way is to just
remove the old config to make sure the new default one is written. There
is no built in mechanism to preserve user changes in karaf.

Christian

On 28.11.2016 14:05, Tim Ward wrote:

I'm trying to work out how to handle configuration of a system
deployed to Karaf.

I can see that configuration items put into etc/.cfg end up being
passed to the @Activate method (or whatever), and that you can change
configuration either by editing the .cfg file or from the Karaf
command line (or, I'm guessing, from a JMX console). So that's all
fine, I think.

The parts of the process that I don't yet understand are

(a) getting the .cfg file into etc in the first place
(b) what happens on upgrade.

Let's say the Java source files for the code are in git, and get built
into bundles using Eclipse, and the bundles are installed into Karaf
by some mechanism (I gather that there are some choices, such as
simply dropping the bundle files into the deploy directory). So the
first question is, how do the initial, default, states of the .cfg
files get from git into the etc directory (I'm hoping for a less error
prone answer than checking them out manually and copying them
manually)?

Then, the life cycle of a configuration file in other contexts is
typically

(1) when the software is first installed, the initial, default state
of the configuration file gets installed at the same time as part of
the same process
(2) the user then edits the configuration file to suit this particular
deployment
(3) a later upgrade to the software comes with a new version of the
configuration file containing some new items, and the upgrade process
must ensure that neither these new items nor the user changes at (2)
get lost.

So how is this managed?

What I've found so far is that one can create a "feature" and use
. But the documentation I've seen doesn't appear to cope
with upgrade - I think it said that new versions of config files would
be silently discarded if an old version was already there? - which
doesn't meet the case (our Operations people get quite cross with
upgrades that do this). At the very very least there needs to be a
clear warning flagged up to the user that they need to do a manual
merge.

And, what is the "URL" that one puts in a , assuming that
there's a solution to the upgrade issue?

To summarise my questions:

(A) What are the options for getting initial, default config files
from git to etc?
(B) How do people cope with the upgrade issue?
(C) If features and  are part of the solution, what's the
"URL"?

Thanks.











--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: Could/should OsgiConfiguration for JAAS fallback to default configuration

[Jean-Baptiste Onofré]

Thanks Sergiy, I will review.

Regards
JB

On 11/28/2016 03:03 PM, Sergiy Shyrkov wrote:

Hello guys,

we've finally tested the fix successfully on our side and have submitted
it as a pull request: https://github.com/apache/karaf/pull/271

Thank you!

Kind regards
Sergiy

On Thu, 20 Oct 2016 08:59:03 +0200, Benjamin Papez
 wrote:


Hi JB,

thanks for looking into it. I already created a jira:
https://issues.apache.org/jira/browse/KARAF-4784

The workaround with an own login module ain't that easy, because we do
not know the names of all current jaas configurations customers have
today (can be multiple per installation). As I understand with OSGI we
(or they) would need to create one realm per such entry name and the
login module would need to be able to read and copy the default
configuration before osgi overrides it. If OsgiConfiguration would
fallback to the default configuration, no action would be required
(for us and for customers migrating to the new relase with Karaf).

Regards,
Benjamin

Am 18.10.2016 um 18:25 schrieb Jean-Baptiste Onofré:

Hi Benjamin,

your proposal makes sense. I will review, create a Jira, and implement.

In the mean time, as a workaround, I think you can provide your own
login module acting as a bypass.

Regards
JB

On 10/18/2016 06:19 PM, Benjamin Papez wrote:

Hello,

we plan to use Karaf embedded in the next version of our Web
Application, which means that we still start the application server
(Tomcat/JBoss/Websphere). Some of our customers are using a JAAS
configuration, mainly Kerberos for SPNEGO. Unfortunately with the step
to use Karaf the current default JAAS configuration is no longer picked
up and used, because Karaf is setting the OsgiConfiguration object into
Configuration.setConfiguration within OsgiConfiguration.init method.

Is it wanted (by design) to ignore all standard/app-server specific
ways
of JAAS configuration?

I would otherwise suggest a modification to OsgiConfiguration, with
something like:

private Configuration defaultConfiguration;

public void init() {
try {
defaultConfiguration = Configuration.getConfiguration();
} catch (RuntimeException ex) {
//
}
Configuration.setConfiguration(this);
}
...
public AppConfigurationEntry[] getAppConfigurationEntry(String
name) {
JaasRealm realm = null;
for (JaasRealm r : realms) {
if (r.getName().equals(name)) {
if (realm == null || r.getRank() > realm.getRank()) {
realm = r;
}
}
}
if (realm != null) {
return realm.getEntries();
} else if (defaultConfiguration != null) {
   return defaultConfiguration.getAppConfigurationEntry(name);
}
return null;
}

public void refresh() {
if (defaultConfiguration != null) {
defaultConfiguration.refresh();
}
}

This way if no OSGI configured JAAS realm can find an
AppConfigurationEntry, we would still try to get it from the default
JAAS configuration, and our customers could keep the same JAAS
configuration as before. Would implementing this suggestion break
anything in Karaf?

Regards,
Benjamin









--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: Configuration file handling?

[Jean-Baptiste Onofré]

Actually, feature  now populate the etc folder as well (since 
Karaf 4.0.5 AFAIR).


Regards
JB

On 11/28/2016 02:50 PM, Christian Schneider wrote:

You already found the confiFile option for features. This is the most
widely used option.
The alternative is the config option which simply adds the config in
config admin but not in etc.

Both variants do not cover the upgrade case. A simply way is to just
remove the old config to make sure the new default one is written. There
is no built in mechanism to preserve user changes in karaf.

Christian

On 28.11.2016 14:05, Tim Ward wrote:

I'm trying to work out how to handle configuration of a system
deployed to Karaf.

I can see that configuration items put into etc/.cfg end up being
passed to the @Activate method (or whatever), and that you can change
configuration either by editing the .cfg file or from the Karaf
command line (or, I'm guessing, from a JMX console). So that's all
fine, I think.

The parts of the process that I don't yet understand are

(a) getting the .cfg file into etc in the first place
(b) what happens on upgrade.

Let's say the Java source files for the code are in git, and get built
into bundles using Eclipse, and the bundles are installed into Karaf
by some mechanism (I gather that there are some choices, such as
simply dropping the bundle files into the deploy directory). So the
first question is, how do the initial, default, states of the .cfg
files get from git into the etc directory (I'm hoping for a less error
prone answer than checking them out manually and copying them manually)?

Then, the life cycle of a configuration file in other contexts is
typically

(1) when the software is first installed, the initial, default state
of the configuration file gets installed at the same time as part of
the same process
(2) the user then edits the configuration file to suit this particular
deployment
(3) a later upgrade to the software comes with a new version of the
configuration file containing some new items, and the upgrade process
must ensure that neither these new items nor the user changes at (2)
get lost.

So how is this managed?

What I've found so far is that one can create a "feature" and use
. But the documentation I've seen doesn't appear to cope
with upgrade - I think it said that new versions of config files would
be silently discarded if an old version was already there? - which
doesn't meet the case (our Operations people get quite cross with
upgrades that do this). At the very very least there needs to be a
clear warning flagged up to the user that they need to do a manual merge.

And, what is the "URL" that one puts in a , assuming that
there's a solution to the upgrade issue?

To summarise my questions:

(A) What are the options for getting initial, default config files
from git to etc?
(B) How do people cope with the upgrade issue?
(C) If features and  are part of the solution, what's the
"URL"?

Thanks.






--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: Configuration file handling?

[Achim Nierbeck]

Hi Tim,

as Christian already stated, you have two different ways of configuring.
1) inisde the feature itself as  element
2) as external location, for example an URI to a resource located in a
Maven Repository. But here you can reference any valid URI pattern where a
corresponding URI handler is registered.

An example for such a use-case may also be found here [1], it's the feature
file for Pax-Web, it references an external file located in a Maven
Repository.

regards, Achim

[1] -
https://github.com/ops4j/org.ops4j.pax.web/blob/master/pax-web-features/src/main/resources/features.xml#L91

2016-11-28 16:22 GMT+01:00 Tim Ward :

> On 28/11/2016 15:14, Christian Schneider wrote:
>
>> On 28.11.2016 15:14, Tim Ward wrote:
>>
>>> Thanks - replies inline:
>>>
>>> On 28/11/2016 13:50, Christian Schneider wrote:
>>>
 You already found the confiFile option for features. This is the most
 widely used option.

>>>
>>> So what is the "URL" in the  element, and how do my config
>>> files get there from git?
>>>
>> The URL typically is a maven url. See https://github.com/apache/kara
>> f-decanter/blob/master/assembly/src/main/feature/feature.xml#L241
>> The default configs are deployed there using build-helper-maven-plugin:
>> https://github.com/apache/karaf-decanter/blob/master/appende
>> r/kafka/pom.xml#L50-L69
>>
>
> Sorry, I'm not sure I get that.
>
> Where, physically, do the config files go? (I've never used a "Maven URL",
> and we use bndtools not Maven for building.)
>
> I was expecting something where the config files were somewhere inside the
> bundle or feature being deployed.
>
> --
> Tim Ward
>
>


-- 

Apache Member
Apache Karaf  Committer & PMC
OPS4J Pax Web  Committer &
Project Lead
blog 
Co-Author of Apache Karaf Cookbook 

Software Architect / Project Manager / Scrum Master

Re: Configuration file handling?

[Guillaume Nodet]

2016-11-28 14:50 GMT+01:00 Christian Schneider :

> You already found the confiFile option for features. This is the most
> widely used option.
> The alternative is the config option which simply adds the config in
> config admin but not in etc.
>

Fwiw, both should add the configuration to the etc/ folder.
The main difference is that  use an external file which can be
any kind of file (an xml, or whatever) while  use an inlined
properties file and write the config both to config admin and to the etc/
folder.


>
> Both variants do not cover the upgrade case. A simply way is to just
> remove the old config to make sure the new default one is written. There is
> no built in mechanism to preserve user changes in karaf.
>
> Christian
>
> On 28.11.2016 14:05, Tim Ward wrote:
>
>> I'm trying to work out how to handle configuration of a system deployed
>> to Karaf.
>>
>> I can see that configuration items put into etc/.cfg end up being
>> passed to the @Activate method (or whatever), and that you can change
>> configuration either by editing the .cfg file or from the Karaf command
>> line (or, I'm guessing, from a JMX console). So that's all fine, I think.
>>
>> The parts of the process that I don't yet understand are
>>
>> (a) getting the .cfg file into etc in the first place
>> (b) what happens on upgrade.
>>
>> Let's say the Java source files for the code are in git, and get built
>> into bundles using Eclipse, and the bundles are installed into Karaf by
>> some mechanism (I gather that there are some choices, such as simply
>> dropping the bundle files into the deploy directory). So the first question
>> is, how do the initial, default, states of the .cfg files get from git into
>> the etc directory (I'm hoping for a less error prone answer than checking
>> them out manually and copying them manually)?
>>
>> Then, the life cycle of a configuration file in other contexts is
>> typically
>>
>> (1) when the software is first installed, the initial, default state of
>> the configuration file gets installed at the same time as part of the same
>> process
>> (2) the user then edits the configuration file to suit this particular
>> deployment
>> (3) a later upgrade to the software comes with a new version of the
>> configuration file containing some new items, and the upgrade process must
>> ensure that neither these new items nor the user changes at (2) get lost.
>>
>> So how is this managed?
>>
>> What I've found so far is that one can create a "feature" and use
>> . But the documentation I've seen doesn't appear to cope with
>> upgrade - I think it said that new versions of config files would be
>> silently discarded if an old version was already there? - which doesn't
>> meet the case (our Operations people get quite cross with upgrades that do
>> this). At the very very least there needs to be a clear warning flagged up
>> to the user that they need to do a manual merge.
>>
>> And, what is the "URL" that one puts in a , assuming that
>> there's a solution to the upgrade issue?
>>
>> To summarise my questions:
>>
>> (A) What are the options for getting initial, default config files from
>> git to etc?
>> (B) How do people cope with the upgrade issue?
>> (C) If features and  are part of the solution, what's the
>> "URL"?
>>
>> Thanks.
>>
>>
>
> --
> Christian Schneider
> http://www.liquid-reality.de
>
> Open Source Architect
> http://www.talend.com
>
>


-- 

Guillaume Nodet

Red Hat, Open Source Integration

Email: gno...@redhat.com
Web: http://fusesource.com
Blog: http://gnodet.blogspot.com/

Re: Configuration file handling?

[Tim Ward]

On 28/11/2016 15:14, Christian Schneider wrote:

On 28.11.2016 15:14, Tim Ward wrote:

Thanks - replies inline:

On 28/11/2016 13:50, Christian Schneider wrote:
You already found the confiFile option for features. This is the 
most widely used option.


So what is the "URL" in the  element, and how do my 
config files get there from git?
The URL typically is a maven url. See 
https://github.com/apache/karaf-decanter/blob/master/assembly/src/main/feature/feature.xml#L241
The default configs are deployed there using 
build-helper-maven-plugin: 
https://github.com/apache/karaf-decanter/blob/master/appender/kafka/pom.xml#L50-L69


Sorry, I'm not sure I get that.

Where, physically, do the config files go? (I've never used a "Maven 
URL", and we use bndtools not Maven for building.)


I was expecting something where the config files were somewhere inside 
the bundle or feature being deployed.


--
Tim Ward

Re: Configuration file handling?

[Christian Schneider]

On 28.11.2016 15:14, Tim Ward wrote:

Thanks - replies inline:

On 28/11/2016 13:50, Christian Schneider wrote:
You already found the confiFile option for features. This is the most 
widely used option.


So what is the "URL" in the  element, and how do my config 
files get there from git?
The URL typically is a maven url. See 
https://github.com/apache/karaf-decanter/blob/master/assembly/src/main/feature/feature.xml#L241
The default configs are deployed there using build-helper-maven-plugin: 
https://github.com/apache/karaf-decanter/blob/master/appender/kafka/pom.xml#L50-L69


The alternative is the config option which simply adds the config in 
config admin but not in etc.


Both variants do not cover the upgrade case. A simply way is to just 
remove the old config to make sure the new default one is written. 
There is no built in mechanism to preserve user changes in karaf.


"just remove the old config" suggests either a manual process (which I 
can't see going down well with Ops) or some non-trivial upgrade 
scripting (which I fear we're going to need anyway). And "just remove 
the old config" doesn't meet the case, as it'll result in a system 
that was working no longer working.


What do people do in practice? - every non-trivial system with a 
lifetime of more than one version must have come across this upgrade 
issue. Write an upgrade script which detects (how?) that an incoming 
feature has an update for a config file, quarantines one version of 
the file somewhere, and tells the user in big red letters that they've 
got some manual merging to do? Do such scripts, or frameworks for 
writing them, exist, to save wheel-reinventing, as this must be a 
common question?

I know two different approaches:
1. Manual upgrades
2. Immutable server pattern: Install scripts that deploy code and the 
actual configuration. In these cases the config is typically not changed 
by the admin.
As such scripts depend a lot on the customer there is no out of the box 
solution.


Christian


--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Re: Configuration file handling?

[Tim Ward]

Thanks - replies inline:

On 28/11/2016 13:50, Christian Schneider wrote:
You already found the confiFile option for features. This is the most 
widely used option.


So what is the "URL" in the  element, and how do my config 
files get there from git?


The alternative is the config option which simply adds the config in 
config admin but not in etc.


Both variants do not cover the upgrade case. A simply way is to just 
remove the old config to make sure the new default one is written. 
There is no built in mechanism to preserve user changes in karaf.


"just remove the old config" suggests either a manual process (which I 
can't see going down well with Ops) or some non-trivial upgrade 
scripting (which I fear we're going to need anyway). And "just remove 
the old config" doesn't meet the case, as it'll result in a system that 
was working no longer working.


What do people do in practice? - every non-trivial system with a 
lifetime of more than one version must have come across this upgrade 
issue. Write an upgrade script which detects (how?) that an incoming 
feature has an update for a config file, quarantines one version of the 
file somewhere, and tells the user in big red letters that they've got 
some manual merging to do? Do such scripts, or frameworks for writing 
them, exist, to save wheel-reinventing, as this must be a common question?



Christian

On 28.11.2016 14:05, Tim Ward wrote:
I'm trying to work out how to handle configuration of a system 
deployed to Karaf.


I can see that configuration items put into etc/.cfg end up 
being passed to the @Activate method (or whatever), and that you can 
change configuration either by editing the .cfg file or from the 
Karaf command line (or, I'm guessing, from a JMX console). So that's 
all fine, I think.


The parts of the process that I don't yet understand are

(a) getting the .cfg file into etc in the first place
(b) what happens on upgrade.

Let's say the Java source files for the code are in git, and get 
built into bundles using Eclipse, and the bundles are installed into 
Karaf by some mechanism (I gather that there are some choices, such 
as simply dropping the bundle files into the deploy directory). So 
the first question is, how do the initial, default, states of the 
.cfg files get from git into the etc directory (I'm hoping for a less 
error prone answer than checking them out manually and copying them 
manually)?


Then, the life cycle of a configuration file in other contexts is 
typically


(1) when the software is first installed, the initial, default state 
of the configuration file gets installed at the same time as part of 
the same process
(2) the user then edits the configuration file to suit this 
particular deployment
(3) a later upgrade to the software comes with a new version of the 
configuration file containing some new items, and the upgrade process 
must ensure that neither these new items nor the user changes at (2) 
get lost.


So how is this managed?

What I've found so far is that one can create a "feature" and use 
. But the documentation I've seen doesn't appear to cope 
with upgrade - I think it said that new versions of config files 
would be silently discarded if an old version was already there? - 
which doesn't meet the case (our Operations people get quite cross 
with upgrades that do this). At the very very least there needs to be 
a clear warning flagged up to the user that they need to do a manual 
merge.


And, what is the "URL" that one puts in a , assuming that 
there's a solution to the upgrade issue?


To summarise my questions:

(A) What are the options for getting initial, default config files 
from git to etc?

(B) How do people cope with the upgrade issue?
(C) If features and  are part of the solution, what's the 
"URL"?


Thanks.







--
Tim Ward

Re: Could/should OsgiConfiguration for JAAS fallback to default configuration

[Sergiy Shyrkov]

Hello guys,

we've finally tested the fix successfully on our side and have submitted  
it as a pull request: https://github.com/apache/karaf/pull/271


Thank you!

Kind regards
Sergiy

On Thu, 20 Oct 2016 08:59:03 +0200, Benjamin Papez   
wrote:



Hi JB,

thanks for looking into it. I already created a jira:  
https://issues.apache.org/jira/browse/KARAF-4784


The workaround with an own login module ain't that easy, because we do  
not know the names of all current jaas configurations customers have  
today (can be multiple per installation). As I understand with OSGI we  
(or they) would need to create one realm per such entry name and the  
login module would need to be able to read and copy the default  
configuration before osgi overrides it. If OsgiConfiguration would  
fallback to the default configuration, no action would be required (for  
us and for customers migrating to the new relase with Karaf).


Regards,
Benjamin

Am 18.10.2016 um 18:25 schrieb Jean-Baptiste Onofré:

Hi Benjamin,

your proposal makes sense. I will review, create a Jira, and implement.

In the mean time, as a workaround, I think you can provide your own  
login module acting as a bypass.


Regards
JB

On 10/18/2016 06:19 PM, Benjamin Papez wrote:

Hello,

we plan to use Karaf embedded in the next version of our Web
Application, which means that we still start the application server
(Tomcat/JBoss/Websphere). Some of our customers are using a JAAS
configuration, mainly Kerberos for SPNEGO. Unfortunately with the step
to use Karaf the current default JAAS configuration is no longer picked
up and used, because Karaf is setting the OsgiConfiguration object into
Configuration.setConfiguration within OsgiConfiguration.init method.

Is it wanted (by design) to ignore all standard/app-server specific  
ways

of JAAS configuration?

I would otherwise suggest a modification to OsgiConfiguration, with
something like:

private Configuration defaultConfiguration;

public void init() {
try {
defaultConfiguration = Configuration.getConfiguration();
} catch (RuntimeException ex) {
//
}
Configuration.setConfiguration(this);
}
...
public AppConfigurationEntry[] getAppConfigurationEntry(String  
name) {

JaasRealm realm = null;
for (JaasRealm r : realms) {
if (r.getName().equals(name)) {
if (realm == null || r.getRank() > realm.getRank()) {
realm = r;
}
}
}
if (realm != null) {
return realm.getEntries();
} else if (defaultConfiguration != null) {
   return defaultConfiguration.getAppConfigurationEntry(name);
}
return null;
}

public void refresh() {
if (defaultConfiguration != null) {
defaultConfiguration.refresh();
}
}

This way if no OSGI configured JAAS realm can find an
AppConfigurationEntry, we would still try to get it from the default
JAAS configuration, and our customers could keep the same JAAS
configuration as before. Would implementing this suggestion break
anything in Karaf?

Regards,
Benjamin







--
Sergiy Shyrkov
Software Architect
Jahia Solutions Group SA
http://www.jahia.com