Author: shuber
Date: Tue Nov 24 15:24:48 2020
New Revision: 1883788
URL: http://svn.apache.org/viewvc?rev=1883788&view=rev
Log:
Site checkin for project Apache Unomi :: Root Project
Added:
unomi/website/manual/1_5_x/images/expression-filtering-layers.png (with
props)
unomi/website/manual/latest/images/expression-filtering-layers.png (with
props)
Modified:
unomi/website/manual/1_1_x/index.html
unomi/website/manual/1_2_x/index.html
unomi/website/manual/1_3_x/index.html
unomi/website/manual/1_4_x/index.html
unomi/website/manual/1_5_x/index.html
unomi/website/manual/latest/index.html
Modified: unomi/website/manual/1_1_x/index.html
URL:
http://svn.apache.org/viewvc/unomi/website/manual/1_1_x/index.html?rev=1883788&r1=1883787&r2=1883788&view=diff
==============================================================================
--- unomi/website/manual/1_1_x/index.html (original)
+++ unomi/website/manual/1_1_x/index.html Tue Nov 24 15:24:48 2020
@@ -1722,7 +1722,7 @@ done by disabling the elasticsearch auto
</div>
<div id="footer">
<div id="footer-text">
-Last updated 2020-05-14 16:06:48 +0200
+Last updated 2020-11-04 10:32:53 +0100
</div>
</div>
</body>
Modified: unomi/website/manual/1_2_x/index.html
URL:
http://svn.apache.org/viewvc/unomi/website/manual/1_2_x/index.html?rev=1883788&r1=1883787&r2=1883788&view=diff
==============================================================================
--- unomi/website/manual/1_2_x/index.html (original)
+++ unomi/website/manual/1_2_x/index.html Tue Nov 24 15:24:48 2020
@@ -2821,7 +2821,7 @@ definition file.</p>
</div>
<div id="footer">
<div id="footer-text">
-Last updated 2020-05-14 16:06:48 +0200
+Last updated 2020-11-04 10:32:53 +0100
</div>
</div>
</body>
Modified: unomi/website/manual/1_3_x/index.html
URL:
http://svn.apache.org/viewvc/unomi/website/manual/1_3_x/index.html?rev=1883788&r1=1883787&r2=1883788&view=diff
==============================================================================
--- unomi/website/manual/1_3_x/index.html (original)
+++ unomi/website/manual/1_3_x/index.html Tue Nov 24 15:24:48 2020
@@ -3401,7 +3401,7 @@ definition file.</p>
</div>
<div id="footer">
<div id="footer-text">
-Last updated 2020-05-14 16:06:48 +0200
+Last updated 2020-11-04 10:32:53 +0100
</div>
</div>
</body>
Modified: unomi/website/manual/1_4_x/index.html
URL:
http://svn.apache.org/viewvc/unomi/website/manual/1_4_x/index.html?rev=1883788&r1=1883787&r2=1883788&view=diff
==============================================================================
--- unomi/website/manual/1_4_x/index.html (original)
+++ unomi/website/manual/1_4_x/index.html Tue Nov 24 15:24:48 2020
@@ -6145,7 +6145,7 @@ They allow to modify an item, that would
</div>
<div id="footer">
<div id="footer-text">
-Last updated 2020-05-14 16:06:48 +0200
+Last updated 2020-11-24 14:42:54 +0100
</div>
</div>
</body>
Added: unomi/website/manual/1_5_x/images/expression-filtering-layers.png
URL:
http://svn.apache.org/viewvc/unomi/website/manual/1_5_x/images/expression-filtering-layers.png?rev=1883788&view=auto
==============================================================================
Binary file - no diff available.
Propchange: unomi/website/manual/1_5_x/images/expression-filtering-layers.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Modified: unomi/website/manual/1_5_x/index.html
URL:
http://svn.apache.org/viewvc/unomi/website/manual/1_5_x/index.html?rev=1883788&r1=1883787&r2=1883788&view=diff
==============================================================================
--- unomi/website/manual/1_5_x/index.html (original)
+++ unomi/website/manual/1_5_x/index.html Tue Nov 24 15:24:48 2020
@@ -72,7 +72,14 @@
<li><a href="#_installing_the_maxmind_geoiplite2_ip_lookup_database">3.5.
Installing the MaxMind GeoIPLite2 IP lookup database</a></li>
<li><a href="#_installing_geonames_database">3.6. Installing Geonames
database</a></li>
<li><a href="#_rest_api_security">3.7. REST API Security</a></li>
-<li><a href="#_scripting_security">3.8. Scripting security</a></li>
+<li><a href="#_scripting_security">3.8. Scripting security</a>
+<ul class="sectlevel3">
+<li><a href="#_multi_layer_scripting_filtering_system">3.8.1. Multi-layer
scripting filtering system</a></li>
+<li><a href="#_scripts_and_expressions">3.8.2. Scripts and expressions</a></li>
+<li><a href="#_scripting_expression_filtering_configuration_parameters">3.8.3.
Scripting expression filtering configuration parameters</a></li>
+<li><a href="#_scripting_roadmap">3.8.4. Scripting roadmap</a></li>
+</ul>
+</li>
<li><a href="#_automatic_profile_merging">3.9. Automatic profile
merging</a></li>
<li><a href="#_securing_a_production_environment">3.10. Securing a production
environment</a></li>
<li><a href="#_integrating_with_an_apache_http_web_server">3.11. Integrating
with an Apache HTTP web server</a></li>
@@ -1825,24 +1832,243 @@ org.ops4j.pax.web.ssl.keypassword=${env:
</div>
<div class="sect2">
<h3 id="_scripting_security">3.8. Scripting security</h3>
+<div class="sect3">
+<h4 id="_multi_layer_scripting_filtering_system">3.8.1. Multi-layer scripting
filtering system</h4>
+<div class="paragraph">
+<p>The scripting security system is multi-layered.</p>
+</div>
+<div class="paragraph">
+<p>For requests coming in through the /context.json endpoint, the following
flow is used to secure incoming requests:</p>
+</div>
+<div class="imageblock">
+<div class="content">
+<img src="images/expression-filtering-layers.png" alt="Expression filtering
layers">
+</div>
+</div>
+<div class="paragraph">
+<p>Conditions submitted through the context.json public endpoint are first
sanitized, meaning that any scripting directly
+injected is removed. However, as conditions can use sub conditions that
include scripting, only the first directly
+injected layer of scripts are removed.</p>
+</div>
+<div class="paragraph">
+<p>The second layer is the expression filtering system, that uses an
allow-listing mechanism to only accept pre-vetted
+expressions (through configuration and deployment on the server side). Any
unrecognized expression will not be accepted.</p>
+</div>
+<div class="paragraph">
+<p>Finally, once the script starts executing in the scripting engine, a
filtering class loader will only let the script
+access classes that have been allowed.</p>
+</div>
+<div class="paragraph">
+<p>This multi-layered approach makes it possible to retain a high level of
security even if one layer is poorly
+configured or abused.</p>
+</div>
+<div class="paragraph">
+<p>For requests coming in through the secure APIs such as rules, only the
condition sanitizing step is skipped,
+otherwise the rest of the filtering system is the same.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_scripts_and_expressions">3.8.2. Scripts and expressions</h4>
+<div class="paragraph">
+<p>Apache Unomi allows using different types of expressions in the following
subsystems:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>context.json filters and personalization queries</p>
+</li>
+<li>
+<p>rule conditions and actions parameters</p>
+</li>
+</ul>
+</div>
<div class="paragraph">
-<p>By default, scripting (using in conditions, segments and rules) is
controlled by a custom classloader that is quite
-restrictive and using a white-list/black list system. It is controlled through
the following property in the
-<code>unomi.custom.system.properties</code> file:</p>
+<p>Apache Unomi uses two integrated scripting languages to provide this
functionality: OGNL and MVEL
+OGNL is deprecated and is now disabled by default in 1.5.2 as it is little
used (and replaced by better performing
+hardcoded property lookups). MVEL is more commonly used in rule actions as in
the following example:</p>
+</div>
+<div class="paragraph">
+<p>From <a
href="https://github.com/apache/unomi/blob/unomi-1.5.x/plugins/baseplugin/src/main/resources/META-INF/cxs/rules/sessionAssigned.json";>https://github.com/apache/unomi/blob/unomi-1.5.x/plugins/baseplugin/src/main/resources/META-INF/cxs/rules/sessionAssigned.json</a>:</p>
</div>
<div class="listingblock">
<div class="content">
-<pre
class="highlight"><code>org.apache.unomi.scripting.allow=${env:UNOMI_ALLOW_SCRIPTING_CLASSES:-org.apache.unomi.api.Event,org.apache.unomi.api.Profile,org.apache.unomi.api.Session,org.apache.unomi.api.Item,org.apache.unomi.api.CustomItem,ognl.*,java.lang.Object,java.util.Map,java.lang.Integer,org.mvel2.*}
-org.apache.unomi.scripting.forbid=${env:UNOMI_FORBID_SCRIPTING_CLASSES:-}</code></pre>
+<pre class="highlight"><code class="language-json" data-lang="json">{
+ "metadata": {
+ "id": "_ajhg9u2s5_sessionAssigned",
+ "name": "Session assigned to a profile",
+ "description": "Update profile visit information",
+ "readOnly":true
+ },
+
+ "condition": {
+ "type": "booleanCondition",
+ "parameterValues": {
+ "subConditions":[
+ {
+ "type": "eventTypeCondition",
+ "parameterValues": {
+ "eventTypeId": "sessionCreated"
+ }
+ },
+ {
+ "type": "eventTypeCondition",
+ "parameterValues": {
+ "eventTypeId": "sessionReassigned"
+ }
+ }
+
+ ],
+ "operator":"or"
+
+ }
+ },
+
+ "actions": [
+ {
+ "parameterValues": {
+ "setPropertyName": "properties.previousVisit",
+ "setPropertyValue": "profileProperty::lastVisit",
+ "storeInSession": false
+ },
+ "type": "setPropertyAction"
+ },
+ {
+ "parameterValues": {
+ "setPropertyName": "properties.lastVisit",
+ "setPropertyValue": "now",
+ "storeInSession": false
+ },
+ "type": "setPropertyAction"
+ },
+ {
+ "parameterValues": {
+ "setPropertyName": "properties.nbOfVisits",
+ "setPropertyValue": "script::profile.properties.?nbOfVisits != null ?
(profile.properties.nbOfVisits + 1) : 1",
+ "storeInSession": false
+ },
+ "type": "setPropertyAction"
+ }
+ ]
+
+}</code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>As we see in the above example, we use an MVEL script with the
setPropertyAction to set a property value.
+Starting with version 1.5.2, any expression use in rules MUST be
allow-listed.</p>
+</div>
+<div class="paragraph">
+<p>OGNL was previously used wherever a parameter could be used, but MVEL could
only be used with a âscript::â prefix.
+Starting with version 1.5.2 OGNL will no longer be allowed and is replaced by
a compatible âhardcodedâ property
+lookup system, while MVEL requires allow-listing the scripts that are to be
used.</p>
+</div>
+<div class="paragraph">
+<p>By default, Apache Unomi comes with some built-in allowed expressions that
cover all the internal uses cases.</p>
+</div>
+<div class="paragraph">
+<p>Default allowed MVEL expressions (from <a
href="https://github.com/apache/unomi/blob/unomi-1.5.x/plugins/baseplugin/src/main/resources/META-INF/cxs/expressions/mvel.json";>https://github.com/apache/unomi/blob/unomi-1.5.x/plugins/baseplugin/src/main/resources/META-INF/cxs/expressions/mvel.json</a>)
:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-json" data-lang="json">[
+ "\\Q'systemProperties.goals.'+goalId+'TargetReached'\\E",
+ "\\Q'now-'+since+'d'\\E",
+ "\\Q'scores.'+scoringPlanId\\E",
+ "\\QminimumDuration*1000\\E",
+ "\\QmaximumDuration*1000\\E",
+ "\\Qprofile.properties.?nbOfVisits != null ? (profile.properties.nbOfVisits
+ 1) : 1\\E",
+ "\\Qsession != null ? session.size + 1 : 0\\E",
+ "\\Q'properties.optimizationTest_'+event.target.itemId\\E",
+ "\\Qevent.target.properties.variantId\\E",
+ "\\Qprofile.properties.?systemProperties.goals.\\E[\\w\\_]*\\QReached !=
null ? (profile.properties.systemProperties.goals.\\E[\\w\\_]*\\QReached) :
'now'\\E",
+ "\\Qprofile.properties.?systemProperties.campaigns.\\E[\\w\\_]*\\QEngaged !=
null ? (profile.properties.systemProperties.campaigns.\\E[\\w\\_]*\\QEngaged) :
'now'\\E"
+]</code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>If you require or are already using custom expressions, you should add a
plugin to Apache Unomi to allow for this.
+The choice of a plugin was to make sure only system administrators and
solution developers could provide such a
+list, avoiding the possibility to provide it through an API call or another
security sensitive deployment mechanism.</p>
</div>
+<div class="paragraph">
+<p>There is another way of allow-listing expressions through configuration,
see the âscripting configuration parametersâ section below.</p>
</div>
<div class="paragraph">
-<p>If you encounter any errors while trying to access a class in a condition
or an action it might be due to this
-restrictive configuration.</p>
+<p>Procedure to add allowed expressions:</p>
</div>
+<div class="olist arabic">
+<ol class="arabic">
+<li>
+<p>Create a new Apache Unomi plugin project.</p>
+</li>
+<li>
+<p>Create a JSON file in src/main/resources/META-INF/cxs/expressions/mvel.json
with an array of regular expressions that will contain the allowed
expressions.</p>
+</li>
+<li>
+<p>Build the project and deploy it to Apache Unomi</p>
+</li>
+</ol>
+</div>
+<div class="paragraph">
+<p>Warning: Do not make regular expressions too general. They should actually
be as specific as possible to avoid potential injection of malicious code.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_scripting_expression_filtering_configuration_parameters">3.8.3.
Scripting expression filtering configuration parameters</h4>
<div class="paragraph">
-<p>If you need, for example when adding a custom item type, to adjust these,
please be careful as scripts may be called
-directly from the context.json personalization conditions and therefore should
be kept minimal.</p>
+<p>Alongside with the allow-listing technology, there are new configuration
parameters to control the security of the scripting engines:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code># These parameters control the list of classes
that are allowed or forbidden when executing expressions.
+org.apache.unomi.scripting.allow=${env:UNOMI_ALLOW_SCRIPTING_CLASSES:-org.apache.unomi.api.Event,org.apache.unomi.api.Profile,org.apache.unomi.api.Session,org.apache.unomi.api.Item,org.apache.unomi.api.CustomItem,ognl.*,java.lang.Object,java.util.Map,java.util.HashMap,java.lang.Integer,org.mvel2.*}
+org.apache.unomi.scripting.forbid=${env:UNOMI_FORBID_SCRIPTING_CLASSES:-}
+
+# This parameter controls the whole expression filtering system. It is not
recommended to turn it off. The main reason to turn it off would be to check if
it is interfering with something, but it should always be active in production.
+org.apache.unomi.scripting.filter.activated=${env:UNOMI_SCRIPTING_FILTER_ACTIVATED:-true}
+
+# The following parameters control the filtering using regular expressions for
each scripting sub-system.
+# The "collections" parameter tells the expression filtering system which
configurations to expect. By default only MVEL and/or OGNL are accepted values,
but in the future these might be replaced by new scripting sub-systems.
+org.apache.unomi.scripting.filter.collections=${env:UNOMI_SCRIPTING_FILTER_COLLECTIONS:-mvel,ognl}
+
+# For each scripting sub-system, there is an allow and a forbid property that
reference a .json files,
+# you can either edit this files or reference your own file directly in the
following config.
+# Note: You can add new expressions to the "allow" file, although it is better
to add them inside any plugins you may be adding.
+# This configuration is only designed to compensate for the cases where
something was not properly designed or to deal with compatibility issues.
+# Just be VERY careful to make your patterns AS SPECIFIC AS POSSIBLE in
order to avoid introducing a way to abuse the expression filtering.
+# Note: It is NOT recommended to change the built-in "forbid" value unless you
are having issues with its value.
+# Note: mvel-allow.json contains an empty array: [], this mean nothing is
allowed, so far.
+# If you want to allow all expression, just remove the property
org.apache.unomi.scripting.filter.mvel.allow, but this is not recommended
+# It's better to list your expressions, and provide them in the
mvel-allow.json file
+# example: ["\\Qsession.size + 1\\E"]
+org.apache.unomi.scripting.filter.mvel.allow=${env:UNOMI_SCRIPTING_FILTER_MVEL_ALLOW:-${karaf.etc}/mvel-allow.json}
+org.apache.unomi.scripting.filter.mvel.forbid=${env:UNOMI_SCRIPTING_FILTER_MVEL_FORBID:-${karaf.etc}/mvel-forbid.json}
+org.apache.unomi.scripting.filter.ognl.allow=${env:UNOMI_SCRIPTING_FILTER_OGNL_ALLOW:-${karaf.etc}/ognl-allow.json}
+org.apache.unomi.scripting.filter.ognl.forbid=${env:UNOMI_SCRIPTING_FILTER_OGNL_FORBID:-${karaf.etc}/ognl-forbid.json}
+
+# This parameter controls whether OGNL scripting is allowed in expressions.
Because of security reasons it is deactivated by default. If you run into
compatibility issues you could reactivate it but it is at your own risk.
+org.apache.unomi.security.properties.useOGNLScripting=${env:UNOMI_SCRIPTING_USE_OGNL:-false}
+
+# This parameter controls the condition sanitizing done on the ContextServlet
(/context.json). If will remove any expressions that start with "script::". It
is not recommended to change this value, unless you run into compatibility
issues.
+org.apache.unomi.security.personalization.sanitizeConditions=${env:UNOMI_SECURITY_SANITIZEPERSONALIZATIONCONDITIONS:-true}</code></pre>
+</div>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_scripting_roadmap">3.8.4. Scripting roadmap</h4>
+<div class="paragraph">
+<p>Scripting will probably undergo major changes in future versions of Apache
Unomi, possibly replacing MVEL with a
+more secure implementation of a scripting language, or possibly even removed
completely (see Groovy actions below).</p>
+</div>
+<div class="paragraph">
+<p>It is recommended that scripting be avoided as in most cases it could be
replaced by custom action implementations,
+which are also possible using the Groovy Scripting language. The main
difference is in the deployment mechanism,
+Groovy actions or Java actions must be deployed as plugins, which require
system administrator access and permissions.</p>
+</div>
+<div class="paragraph">
+<p>These changes will not happen on maintenance versions of Apache Unomi, only
in the next major version. Maintenance
+versions will of course maintain compatibility with existing scripting
solutions.</p>
+</div>
</div>
</div>
<div class="sect2">
@@ -6396,7 +6622,7 @@ only be sent if the user has authenticat
</div>
<div class="listingblock">
<div class="content">
-<pre class="highlight"><code>bundle:install
mvn:org.apache.unomi/login-integration-samples/${project.version}</code></pre>
+<pre class="highlight"><code>bundle:install
mvn:org.apache.unomi/login-integration-sample/${project.version}</code></pre>
</div>
</div>
<div class="paragraph">
@@ -8747,7 +8973,7 @@ They allow to modify an item, that would
</div>
<div id="footer">
<div id="footer-text">
-Last updated 2020-06-05 17:34:24 +0200
+Last updated 2020-11-24 14:42:54 +0100
</div>
</div>
</body>
Added: unomi/website/manual/latest/images/expression-filtering-layers.png
URL:
http://svn.apache.org/viewvc/unomi/website/manual/latest/images/expression-filtering-layers.png?rev=1883788&view=auto
==============================================================================
Binary file - no diff available.
Propchange: unomi/website/manual/latest/images/expression-filtering-layers.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Modified: unomi/website/manual/latest/index.html
URL:
http://svn.apache.org/viewvc/unomi/website/manual/latest/index.html?rev=1883788&r1=1883787&r2=1883788&view=diff
==============================================================================
--- unomi/website/manual/latest/index.html (original)
+++ unomi/website/manual/latest/index.html Tue Nov 24 15:24:48 2020
@@ -72,7 +72,14 @@
<li><a href="#_installing_the_maxmind_geoiplite2_ip_lookup_database">3.5.
Installing the MaxMind GeoIPLite2 IP lookup database</a></li>
<li><a href="#_installing_geonames_database">3.6. Installing Geonames
database</a></li>
<li><a href="#_rest_api_security">3.7. REST API Security</a></li>
-<li><a href="#_scripting_security">3.8. Scripting security</a></li>
+<li><a href="#_scripting_security">3.8. Scripting security</a>
+<ul class="sectlevel3">
+<li><a href="#_multi_layer_scripting_filtering_system">3.8.1. Multi-layer
scripting filtering system</a></li>
+<li><a href="#_scripts_and_expressions">3.8.2. Scripts and expressions</a></li>
+<li><a href="#_scripting_expression_filtering_configuration_parameters">3.8.3.
Scripting expression filtering configuration parameters</a></li>
+<li><a href="#_scripting_roadmap">3.8.4. Scripting roadmap</a></li>
+</ul>
+</li>
<li><a href="#_automatic_profile_merging">3.9. Automatic profile
merging</a></li>
<li><a href="#_securing_a_production_environment">3.10. Securing a production
environment</a></li>
<li><a href="#_integrating_with_an_apache_http_web_server">3.11. Integrating
with an Apache HTTP web server</a></li>
@@ -1825,24 +1832,243 @@ org.ops4j.pax.web.ssl.keypassword=${env:
</div>
<div class="sect2">
<h3 id="_scripting_security">3.8. Scripting security</h3>
+<div class="sect3">
+<h4 id="_multi_layer_scripting_filtering_system">3.8.1. Multi-layer scripting
filtering system</h4>
+<div class="paragraph">
+<p>The scripting security system is multi-layered.</p>
+</div>
+<div class="paragraph">
+<p>For requests coming in through the /context.json endpoint, the following
flow is used to secure incoming requests:</p>
+</div>
+<div class="imageblock">
+<div class="content">
+<img src="images/expression-filtering-layers.png" alt="Expression filtering
layers">
+</div>
+</div>
+<div class="paragraph">
+<p>Conditions submitted through the context.json public endpoint are first
sanitized, meaning that any scripting directly
+injected is removed. However, as conditions can use sub conditions that
include scripting, only the first directly
+injected layer of scripts are removed.</p>
+</div>
+<div class="paragraph">
+<p>The second layer is the expression filtering system, that uses an
allow-listing mechanism to only accept pre-vetted
+expressions (through configuration and deployment on the server side). Any
unrecognized expression will not be accepted.</p>
+</div>
+<div class="paragraph">
+<p>Finally, once the script starts executing in the scripting engine, a
filtering class loader will only let the script
+access classes that have been allowed.</p>
+</div>
+<div class="paragraph">
+<p>This multi-layered approach makes it possible to retain a high level of
security even if one layer is poorly
+configured or abused.</p>
+</div>
+<div class="paragraph">
+<p>For requests coming in through the secure APIs such as rules, only the
condition sanitizing step is skipped,
+otherwise the rest of the filtering system is the same.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_scripts_and_expressions">3.8.2. Scripts and expressions</h4>
+<div class="paragraph">
+<p>Apache Unomi allows using different types of expressions in the following
subsystems:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>context.json filters and personalization queries</p>
+</li>
+<li>
+<p>rule conditions and actions parameters</p>
+</li>
+</ul>
+</div>
<div class="paragraph">
-<p>By default, scripting (using in conditions, segments and rules) is
controlled by a custom classloader that is quite
-restrictive and using a white-list/black list system. It is controlled through
the following property in the
-<code>unomi.custom.system.properties</code> file:</p>
+<p>Apache Unomi uses two integrated scripting languages to provide this
functionality: OGNL and MVEL
+OGNL is deprecated and is now disabled by default in 1.5.2 as it is little
used (and replaced by better performing
+hardcoded property lookups). MVEL is more commonly used in rule actions as in
the following example:</p>
+</div>
+<div class="paragraph">
+<p>From <a
href="https://github.com/apache/unomi/blob/unomi-1.5.x/plugins/baseplugin/src/main/resources/META-INF/cxs/rules/sessionAssigned.json";>https://github.com/apache/unomi/blob/unomi-1.5.x/plugins/baseplugin/src/main/resources/META-INF/cxs/rules/sessionAssigned.json</a>:</p>
</div>
<div class="listingblock">
<div class="content">
-<pre
class="highlight"><code>org.apache.unomi.scripting.allow=${env:UNOMI_ALLOW_SCRIPTING_CLASSES:-org.apache.unomi.api.Event,org.apache.unomi.api.Profile,org.apache.unomi.api.Session,org.apache.unomi.api.Item,org.apache.unomi.api.CustomItem,ognl.*,java.lang.Object,java.util.Map,java.lang.Integer,org.mvel2.*}
-org.apache.unomi.scripting.forbid=${env:UNOMI_FORBID_SCRIPTING_CLASSES:-}</code></pre>
+<pre class="highlight"><code class="language-json" data-lang="json">{
+ "metadata": {
+ "id": "_ajhg9u2s5_sessionAssigned",
+ "name": "Session assigned to a profile",
+ "description": "Update profile visit information",
+ "readOnly":true
+ },
+
+ "condition": {
+ "type": "booleanCondition",
+ "parameterValues": {
+ "subConditions":[
+ {
+ "type": "eventTypeCondition",
+ "parameterValues": {
+ "eventTypeId": "sessionCreated"
+ }
+ },
+ {
+ "type": "eventTypeCondition",
+ "parameterValues": {
+ "eventTypeId": "sessionReassigned"
+ }
+ }
+
+ ],
+ "operator":"or"
+
+ }
+ },
+
+ "actions": [
+ {
+ "parameterValues": {
+ "setPropertyName": "properties.previousVisit",
+ "setPropertyValue": "profileProperty::lastVisit",
+ "storeInSession": false
+ },
+ "type": "setPropertyAction"
+ },
+ {
+ "parameterValues": {
+ "setPropertyName": "properties.lastVisit",
+ "setPropertyValue": "now",
+ "storeInSession": false
+ },
+ "type": "setPropertyAction"
+ },
+ {
+ "parameterValues": {
+ "setPropertyName": "properties.nbOfVisits",
+ "setPropertyValue": "script::profile.properties.?nbOfVisits != null ?
(profile.properties.nbOfVisits + 1) : 1",
+ "storeInSession": false
+ },
+ "type": "setPropertyAction"
+ }
+ ]
+
+}</code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>As we see in the above example, we use an MVEL script with the
setPropertyAction to set a property value.
+Starting with version 1.5.2, any expression use in rules MUST be
allow-listed.</p>
+</div>
+<div class="paragraph">
+<p>OGNL was previously used wherever a parameter could be used, but MVEL could
only be used with a âscript::â prefix.
+Starting with version 1.5.2 OGNL will no longer be allowed and is replaced by
a compatible âhardcodedâ property
+lookup system, while MVEL requires allow-listing the scripts that are to be
used.</p>
+</div>
+<div class="paragraph">
+<p>By default, Apache Unomi comes with some built-in allowed expressions that
cover all the internal uses cases.</p>
+</div>
+<div class="paragraph">
+<p>Default allowed MVEL expressions (from <a
href="https://github.com/apache/unomi/blob/unomi-1.5.x/plugins/baseplugin/src/main/resources/META-INF/cxs/expressions/mvel.json";>https://github.com/apache/unomi/blob/unomi-1.5.x/plugins/baseplugin/src/main/resources/META-INF/cxs/expressions/mvel.json</a>)
:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-json" data-lang="json">[
+ "\\Q'systemProperties.goals.'+goalId+'TargetReached'\\E",
+ "\\Q'now-'+since+'d'\\E",
+ "\\Q'scores.'+scoringPlanId\\E",
+ "\\QminimumDuration*1000\\E",
+ "\\QmaximumDuration*1000\\E",
+ "\\Qprofile.properties.?nbOfVisits != null ? (profile.properties.nbOfVisits
+ 1) : 1\\E",
+ "\\Qsession != null ? session.size + 1 : 0\\E",
+ "\\Q'properties.optimizationTest_'+event.target.itemId\\E",
+ "\\Qevent.target.properties.variantId\\E",
+ "\\Qprofile.properties.?systemProperties.goals.\\E[\\w\\_]*\\QReached !=
null ? (profile.properties.systemProperties.goals.\\E[\\w\\_]*\\QReached) :
'now'\\E",
+ "\\Qprofile.properties.?systemProperties.campaigns.\\E[\\w\\_]*\\QEngaged !=
null ? (profile.properties.systemProperties.campaigns.\\E[\\w\\_]*\\QEngaged) :
'now'\\E"
+]</code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>If you require or are already using custom expressions, you should add a
plugin to Apache Unomi to allow for this.
+The choice of a plugin was to make sure only system administrators and
solution developers could provide such a
+list, avoiding the possibility to provide it through an API call or another
security sensitive deployment mechanism.</p>
</div>
+<div class="paragraph">
+<p>There is another way of allow-listing expressions through configuration,
see the âscripting configuration parametersâ section below.</p>
</div>
<div class="paragraph">
-<p>If you encounter any errors while trying to access a class in a condition
or an action it might be due to this
-restrictive configuration.</p>
+<p>Procedure to add allowed expressions:</p>
</div>
+<div class="olist arabic">
+<ol class="arabic">
+<li>
+<p>Create a new Apache Unomi plugin project.</p>
+</li>
+<li>
+<p>Create a JSON file in src/main/resources/META-INF/cxs/expressions/mvel.json
with an array of regular expressions that will contain the allowed
expressions.</p>
+</li>
+<li>
+<p>Build the project and deploy it to Apache Unomi</p>
+</li>
+</ol>
+</div>
+<div class="paragraph">
+<p>Warning: Do not make regular expressions too general. They should actually
be as specific as possible to avoid potential injection of malicious code.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_scripting_expression_filtering_configuration_parameters">3.8.3.
Scripting expression filtering configuration parameters</h4>
<div class="paragraph">
-<p>If you need, for example when adding a custom item type, to adjust these,
please be careful as scripts may be called
-directly from the context.json personalization conditions and therefore should
be kept minimal.</p>
+<p>Alongside with the allow-listing technology, there are new configuration
parameters to control the security of the scripting engines:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code># These parameters control the list of classes
that are allowed or forbidden when executing expressions.
+org.apache.unomi.scripting.allow=${env:UNOMI_ALLOW_SCRIPTING_CLASSES:-org.apache.unomi.api.Event,org.apache.unomi.api.Profile,org.apache.unomi.api.Session,org.apache.unomi.api.Item,org.apache.unomi.api.CustomItem,ognl.*,java.lang.Object,java.util.Map,java.util.HashMap,java.lang.Integer,org.mvel2.*}
+org.apache.unomi.scripting.forbid=${env:UNOMI_FORBID_SCRIPTING_CLASSES:-}
+
+# This parameter controls the whole expression filtering system. It is not
recommended to turn it off. The main reason to turn it off would be to check if
it is interfering with something, but it should always be active in production.
+org.apache.unomi.scripting.filter.activated=${env:UNOMI_SCRIPTING_FILTER_ACTIVATED:-true}
+
+# The following parameters control the filtering using regular expressions for
each scripting sub-system.
+# The "collections" parameter tells the expression filtering system which
configurations to expect. By default only MVEL and/or OGNL are accepted values,
but in the future these might be replaced by new scripting sub-systems.
+org.apache.unomi.scripting.filter.collections=${env:UNOMI_SCRIPTING_FILTER_COLLECTIONS:-mvel,ognl}
+
+# For each scripting sub-system, there is an allow and a forbid property that
reference a .json files,
+# you can either edit this files or reference your own file directly in the
following config.
+# Note: You can add new expressions to the "allow" file, although it is better
to add them inside any plugins you may be adding.
+# This configuration is only designed to compensate for the cases where
something was not properly designed or to deal with compatibility issues.
+# Just be VERY careful to make your patterns AS SPECIFIC AS POSSIBLE in
order to avoid introducing a way to abuse the expression filtering.
+# Note: It is NOT recommended to change the built-in "forbid" value unless you
are having issues with its value.
+# Note: mvel-allow.json contains an empty array: [], this mean nothing is
allowed, so far.
+# If you want to allow all expression, just remove the property
org.apache.unomi.scripting.filter.mvel.allow, but this is not recommended
+# It's better to list your expressions, and provide them in the
mvel-allow.json file
+# example: ["\\Qsession.size + 1\\E"]
+org.apache.unomi.scripting.filter.mvel.allow=${env:UNOMI_SCRIPTING_FILTER_MVEL_ALLOW:-${karaf.etc}/mvel-allow.json}
+org.apache.unomi.scripting.filter.mvel.forbid=${env:UNOMI_SCRIPTING_FILTER_MVEL_FORBID:-${karaf.etc}/mvel-forbid.json}
+org.apache.unomi.scripting.filter.ognl.allow=${env:UNOMI_SCRIPTING_FILTER_OGNL_ALLOW:-${karaf.etc}/ognl-allow.json}
+org.apache.unomi.scripting.filter.ognl.forbid=${env:UNOMI_SCRIPTING_FILTER_OGNL_FORBID:-${karaf.etc}/ognl-forbid.json}
+
+# This parameter controls whether OGNL scripting is allowed in expressions.
Because of security reasons it is deactivated by default. If you run into
compatibility issues you could reactivate it but it is at your own risk.
+org.apache.unomi.security.properties.useOGNLScripting=${env:UNOMI_SCRIPTING_USE_OGNL:-false}
+
+# This parameter controls the condition sanitizing done on the ContextServlet
(/context.json). If will remove any expressions that start with "script::". It
is not recommended to change this value, unless you run into compatibility
issues.
+org.apache.unomi.security.personalization.sanitizeConditions=${env:UNOMI_SECURITY_SANITIZEPERSONALIZATIONCONDITIONS:-true}</code></pre>
+</div>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_scripting_roadmap">3.8.4. Scripting roadmap</h4>
+<div class="paragraph">
+<p>Scripting will probably undergo major changes in future versions of Apache
Unomi, possibly replacing MVEL with a
+more secure implementation of a scripting language, or possibly even removed
completely (see Groovy actions below).</p>
+</div>
+<div class="paragraph">
+<p>It is recommended that scripting be avoided as in most cases it could be
replaced by custom action implementations,
+which are also possible using the Groovy Scripting language. The main
difference is in the deployment mechanism,
+Groovy actions or Java actions must be deployed as plugins, which require
system administrator access and permissions.</p>
+</div>
+<div class="paragraph">
+<p>These changes will not happen on maintenance versions of Apache Unomi, only
in the next major version. Maintenance
+versions will of course maintain compatibility with existing scripting
solutions.</p>
+</div>
</div>
</div>
<div class="sect2">
@@ -6396,7 +6622,7 @@ only be sent if the user has authenticat
</div>
<div class="listingblock">
<div class="content">
-<pre class="highlight"><code>bundle:install
mvn:org.apache.unomi/login-integration-samples/${project.version}</code></pre>
+<pre class="highlight"><code>bundle:install
mvn:org.apache.unomi/login-integration-sample/${project.version}</code></pre>
</div>
</div>
<div class="paragraph">
@@ -8931,7 +9157,7 @@ They allow to modify an item, that would
</div>
<div id="footer">
<div id="footer-text">
-Last updated 2020-06-05 17:34:24 +0200
+Last updated 2020-11-24 14:42:54 +0100
</div>
</div>
</body>
