Added: unomi/website/manual/1_6_x/index.html URL: http://svn.apache.org/viewvc/unomi/website/manual/1_6_x/index.html?rev=1898498&view=auto ============================================================================== --- unomi/website/manual/1_6_x/index.html (added) +++ unomi/website/manual/1_6_x/index.html Tue Mar 1 08:17:59 2022 @@ -0,0 +1,9408 @@ +<!DOCTYPE html> +<html lang="en"> +<head> +<meta charset="UTF-8"> +<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]--> +<meta name="viewport" content="width=device-width, initial-scale=1.0"> +<meta name="generator" content="Asciidoctor 1.5.8"> +<meta name="author" content="Apache Software Foundation"> +<title>Apache Unomi 1.x - Documentation</title> +<link rel="stylesheet" href="./apache.css"> +</head> +<body class="article toc2 toc-left"> +<div id="header"> +<h1>Apache Unomi 1.x - Documentation</h1> +<div class="details"> +<span id="author" class="author">Apache Software Foundation</span><br> +</div> +<div id="toc" class="toc2"> +<div id="toctitle">Table of Contents</div> +<ul class="sectlevel1"> +<li><a href="#_quick_start">1. Quick start</a> +<ul class="sectlevel2"> +<li><a href="#_five_minutes_quickstart">1.1. Five Minutes QuickStart</a></li> +</ul> +</li> +<li><a href="#_first_steps_with_apache_unomi">2. First steps with Apache Unomi</a> +<ul class="sectlevel2"> +<li><a href="#_getting_started_with_unomi">2.1. Getting started with Unomi</a> +<ul class="sectlevel3"> +<li><a href="#_prerequisites">2.1.1. Prerequisites</a></li> +<li><a href="#_running_unomi">2.1.2. Running Unomi</a></li> +</ul> +</li> +<li><a href="#_recipes">2.2. Recipes</a> +<ul class="sectlevel3"> +<li><a href="#_introduction">2.2.1. Introduction</a></li> +<li><a href="#_how_to_read_a_profile">2.2.2. How to read a profile</a></li> +<li><a href="#_how_to_update_a_profile_from_the_public_internet">2.2.3. How to update a profile from the public internet</a></li> +<li><a href="#_how_to_search_for_profile_events">2.2.4. How to search for profile events</a></li> +<li><a href="#_how_to_create_a_new_rule">2.2.5. How to create a new rule</a></li> +<li><a href="#_how_to_search_for_profiles">2.2.6. How to search for profiles</a></li> +<li><a href="#_getting_updating_consents">2.2.7. Getting / updating consents</a></li> +<li><a href="#_how_to_send_a_login_event_to_unomi">2.2.8. How to send a login event to Unomi</a></li> +</ul> +</li> +<li><a href="#_request_examples">2.3. Request examples</a> +<ul class="sectlevel3"> +<li><a href="#_retrieving_your_first_context">2.3.1. Retrieving your first context</a></li> +<li><a href="#_retrieving_a_context_as_a_json_object">2.3.2. Retrieving a context as a JSON object.</a></li> +<li><a href="#_accessing_profile_properties_in_a_context">2.3.3. Accessing profile properties in a context</a></li> +<li><a href="#_sending_events_using_the_context_servlet">2.3.4. Sending events using the context servlet</a></li> +<li><a href="#_sending_events_using_the_eventcollector_servlet">2.3.5. Sending events using the eventcollector servlet</a></li> +<li><a href="#_where_to_go_from_here">2.3.6. Where to go from here</a></li> +</ul> +</li> +<li><a href="#_web_tracker">2.4. Web Tracker</a> +<ul class="sectlevel3"> +<li><a href="#_getting_started">2.4.1. Getting started</a></li> +<li><a href="#_how_to_contribute">2.4.2. How to contribute</a></li> +<li><a href="#_tracking_page_views">2.4.3. Tracking page views</a></li> +<li><a href="#_tracking_form_submissions">2.4.4. Tracking form submissions</a></li> +</ul> +</li> +</ul> +</li> +<li><a href="#_configuration">3. Configuration</a> +<ul class="sectlevel2"> +<li><a href="#_centralized_configuration">3.1. Centralized configuration</a></li> +<li><a href="#_changing_the_default_configuration_using_environment_variables_i_e_docker_configuration">3.2. Changing the default configuration using environment variables (i.e. Docker configuration)</a></li> +<li><a href="#_changing_the_default_configuration_using_property_files">3.3. Changing the default configuration using property files</a></li> +<li><a href="#_secured_events_configuration">3.4. Secured events configuration</a></li> +<li><a href="#_installing_the_maxmind_geoiplite2_ip_lookup_database">3.5. Installing the MaxMind GeoIPLite2 IP lookup database</a></li> +<li><a href="#_installing_geonames_database">3.6. Installing Geonames database</a></li> +<li><a href="#_rest_api_security">3.7. REST API Security</a></li> +<li><a href="#_scripting_security">3.8. Scripting security</a> +<ul class="sectlevel3"> +<li><a href="#_multi_layer_scripting_filtering_system">3.8.1. Multi-layer scripting filtering system</a></li> +<li><a href="#_scripts_and_expressions">3.8.2. Scripts and expressions</a></li> +<li><a href="#_scripting_expression_filtering_configuration_parameters">3.8.3. Scripting expression filtering configuration parameters</a></li> +<li><a href="#_groovy_actions">3.8.4. Groovy Actions</a></li> +<li><a href="#_scripting_roadmap">3.8.5. Scripting roadmap</a></li> +</ul> +</li> +<li><a href="#_automatic_profile_merging">3.9. Automatic profile merging</a></li> +<li><a href="#_securing_a_production_environment">3.10. Securing a production environment</a></li> +<li><a href="#_integrating_with_an_apache_http_web_server">3.11. Integrating with an Apache HTTP web server</a></li> +<li><a href="#_changing_the_default_tracking_location">3.12. Changing the default tracking location</a></li> +<li><a href="#_apache_karaf_ssh_console">3.13. Apache Karaf SSH Console</a></li> +<li><a href="#_elasticsearch_authentication_and_security">3.14. ElasticSearch authentication and security</a> +<ul class="sectlevel3"> +<li><a href="#_user_authentication">3.14.1. User authentication !</a></li> +<li><a href="#_ssl_communication">3.14.2. SSL communication</a></li> +</ul> +</li> +</ul> +</li> +<li><a href="#_migrations">4. Migrations</a> +<ul class="sectlevel2"> +<li><a href="#_from_version_1_4_to_1_5">4.1. From version 1.4 to 1.5</a> +<ul class="sectlevel3"> +<li><a href="#_data_model_and_elasticsearch_7">4.1.1. Data model and ElasticSearch 7</a></li> +<li><a href="#_api_changes">4.1.2. API changes</a></li> +<li><a href="#_migration_steps">4.1.3. Migration steps</a></li> +</ul> +</li> +<li><a href="#_important_changes_in_public_servlets_since_version_1_5_5_and_2_0_0">4.2. Important changes in public servlets since version 1.5.5 and 2.0.0</a></li> +</ul> +</li> +<li><a href="#_queries_and_aggregations">5. Queries and aggregations</a> +<ul class="sectlevel2"> +<li><a href="#_query_counts">5.1. Query counts</a></li> +<li><a href="#_metrics">5.2. Metrics</a></li> +<li><a href="#_aggregations">5.3. Aggregations</a> +<ul class="sectlevel3"> +<li><a href="#_aggregation_types">5.3.1. Aggregation types</a></li> +</ul> +</li> +</ul> +</li> +<li><a href="#_profile_import_export">6. Profile import & export</a> +<ul class="sectlevel2"> +<li><a href="#_importing_profiles">6.1. Importing profiles</a> +<ul class="sectlevel3"> +<li><a href="#_import_api">6.1.1. Import API</a></li> +</ul> +</li> +<li><a href="#_exporting_profiles">6.2. Exporting profiles</a> +<ul class="sectlevel3"> +<li><a href="#_export_api">6.2.1. Export API</a></li> +</ul> +</li> +<li><a href="#_configuration_in_details">6.3. Configuration in details</a></li> +</ul> +</li> +<li><a href="#_consent_management">7. Consent management</a> +<ul class="sectlevel2"> +<li><a href="#_consent_api">7.1. Consent API</a> +<ul class="sectlevel3"> +<li><a href="#_profiles_with_consents">7.1.1. Profiles with consents</a></li> +<li><a href="#_consent_type_definitions">7.1.2. Consent type definitions</a></li> +<li><a href="#_creating_update_a_visitor_consent">7.1.3. Creating / update a visitor consent</a></li> +<li><a href="#_how_it_works_internally">7.1.4. How it works (internally)</a></li> +</ul> +</li> +</ul> +</li> +<li><a href="#_privacy_management">8. Privacy management</a> +<ul class="sectlevel2"> +<li><a href="#_setting_up_access_to_the_privacy_endpoint">8.1. Setting up access to the privacy endpoint</a></li> +<li><a href="#_anonymizing_a_profile">8.2. Anonymizing a profile</a></li> +<li><a href="#_downloading_profile_data">8.3. Downloading profile data</a></li> +<li><a href="#_deleting_a_profile">8.4. Deleting a profile</a></li> +<li><a href="#_related">8.5. Related</a></li> +</ul> +</li> +<li><a href="#_cluster_setup">9. Cluster setup</a> +<ul class="sectlevel2"> +<li><a href="#_cluster_setup_2">9.1. Cluster setup</a></li> +</ul> +</li> +<li><a href="#_reference">10. Reference</a> +<ul class="sectlevel2"> +<li><a href="#_useful_apache_unomi_urls">10.1. Useful Apache Unomi URLs</a></li> +<li><a href="#_how_profile_tracking_works">10.2. How profile tracking works</a> +<ul class="sectlevel3"> +<li><a href="#_steps">10.2.1. Steps</a></li> +</ul> +</li> +<li><a href="#_context_request_flow">10.3. Context Request Flow</a></li> +<li><a href="#_data_model_overview">10.4. Data Model Overview</a></li> +<li><a href="#_scope">10.5. Scope</a> +<ul class="sectlevel3"> +<li><a href="#_example">10.5.1. Example</a></li> +</ul> +</li> +<li><a href="#_item">10.6. Item</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition">10.6.1. Structure definition</a></li> +</ul> +</li> +<li><a href="#_metadata">10.7. Metadata</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_2">10.7.1. Structure definition</a></li> +<li><a href="#_example_2">10.7.2. Example</a></li> +</ul> +</li> +<li><a href="#_metadataitem">10.8. MetadataItem</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_3">10.8.1. Structure definition</a></li> +<li><a href="#_example_3">10.8.2. Example</a></li> +</ul> +</li> +<li><a href="#_event">10.9. Event</a> +<ul class="sectlevel3"> +<li><a href="#_fields">10.9.1. Fields</a></li> +<li><a href="#_event_types">10.9.2. Event types</a></li> +</ul> +</li> +<li><a href="#_profile">10.10. Profile</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_4">10.10.1. Structure definition</a></li> +<li><a href="#_example_4">10.10.2. Example</a></li> +</ul> +</li> +<li><a href="#_persona">10.11. Persona</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_5">10.11.1. Structure definition</a></li> +<li><a href="#_example_5">10.11.2. Example</a></li> +</ul> +</li> +<li><a href="#_consent">10.12. Consent</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_6">10.12.1. Structure definition</a></li> +<li><a href="#_example_6">10.12.2. Example</a></li> +</ul> +</li> +<li><a href="#_session">10.13. Session</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_7">10.13.1. Structure definition</a></li> +<li><a href="#_example_7">10.13.2. Example</a></li> +</ul> +</li> +<li><a href="#_segment">10.14. Segment</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_8">10.14.1. Structure definition</a></li> +<li><a href="#_example_8">10.14.2. Example</a></li> +</ul> +</li> +<li><a href="#_condition">10.15. Condition</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_9">10.15.1. Structure definition</a></li> +<li><a href="#_example_9">10.15.2. Example</a></li> +</ul> +</li> +<li><a href="#_rule">10.16. Rule</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_10">10.16.1. Structure definition</a></li> +<li><a href="#_example_10">10.16.2. Example</a></li> +</ul> +</li> +<li><a href="#_action">10.17. Action</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_11">10.17.1. Structure definition</a></li> +<li><a href="#_example_11">10.17.2. Example</a></li> +</ul> +</li> +<li><a href="#_list">10.18. List</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_12">10.18.1. Structure definition</a></li> +<li><a href="#_example_12">10.18.2. Example</a></li> +</ul> +</li> +<li><a href="#_goal">10.19. Goal</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_13">10.19.1. Structure definition</a></li> +<li><a href="#_example_13">10.19.2. Example</a></li> +</ul> +</li> +<li><a href="#_campaign">10.20. Campaign</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_14">10.20.1. Structure definition</a></li> +<li><a href="#_example_14">10.20.2. Example</a></li> +</ul> +</li> +<li><a href="#_scoring_plan">10.21. Scoring plan</a> +<ul class="sectlevel3"> +<li><a href="#_structure_definition_15">10.21.1. Structure definition</a></li> +<li><a href="#_example_15">10.21.2. Example</a></li> +</ul> +</li> +<li><a href="#_built_in_event_types">10.22. Built-in Event types</a> +<ul class="sectlevel3"> +<li><a href="#_login_event_type">10.22.1. Login event type</a></li> +<li><a href="#_view_event_type">10.22.2. View event type</a></li> +<li><a href="#_form_event_type">10.22.3. Form event type</a></li> +<li><a href="#_update_properties_event_type">10.22.4. Update properties event type</a></li> +<li><a href="#_identify_event_type">10.22.5. Identify event type</a></li> +<li><a href="#_session_created_event_type">10.22.6. Session created event type</a></li> +<li><a href="#_goal_event_type">10.22.7. Goal event type</a></li> +<li><a href="#_modify_consent_event_type">10.22.8. Modify consent event type</a></li> +</ul> +</li> +<li><a href="#_built_in_condition_types">10.23. Built-in condition types</a> +<ul class="sectlevel3"> +<li><a href="#_existing_condition_type_descriptors">10.23.1. Existing condition type descriptors</a></li> +</ul> +</li> +<li><a href="#_built_in_action_types">10.24. Built-in action types</a> +<ul class="sectlevel3"> +<li><a href="#_existing_action_types_descriptors">10.24.1. Existing action types descriptors</a></li> +</ul> +</li> +<li><a href="#_updating_events_using_the_context_servlet">10.25. Updating Events Using the Context Servlet</a> +<ul class="sectlevel3"> +<li><a href="#_solution">10.25.1. Solution</a></li> +<li><a href="#_defining_rules">10.25.2. Defining Rules</a></li> +</ul> +</li> +</ul> +</li> +<li><a href="#_integration_samples">11. Integration samples</a> +<ul class="sectlevel2"> +<li><a href="#_samples">11.1. Samples</a></li> +<li><a href="#_login_sample">11.2. Login sample</a> +<ul class="sectlevel3"> +<li><a href="#_warning">11.2.1. Warning !</a></li> +<li><a href="#_installing_the_samples">11.2.2. Installing the samples</a></li> +</ul> +</li> +<li><a href="#_twitter_sample">11.3. Twitter sample</a> +<ul class="sectlevel3"> +<li><a href="#_overview">11.3.1. Overview</a></li> +<li><a href="#_interacting_with_the_context_server">11.3.2. Interacting with the context server</a></li> +<li><a href="#_retrieving_context_information_from_unomi_using_the_context_servlet">11.3.3. Retrieving context information from Unomi using the context servlet</a></li> +</ul> +</li> +<li><a href="#_example_24">11.4. Example</a> +<ul class="sectlevel3"> +<li><a href="#_html_page">11.4.1. HTML page</a></li> +<li><a href="#_javascript">11.4.2. Javascript</a></li> +</ul> +</li> +<li><a href="#_conclusion">11.5. Conclusion</a></li> +<li><a href="#_annex">11.6. Annex</a></li> +<li><a href="#_weather_update_sample">11.7. Weather update sample</a></li> +</ul> +</li> +<li><a href="#_connectors">12. Connectors</a> +<ul class="sectlevel2"> +<li><a href="#_connectors_2">12.1. Connectors</a> +<ul class="sectlevel3"> +<li><a href="#_call_for_contributors">12.1.1. Call for contributors</a></li> +</ul> +</li> +<li><a href="#_salesforce_connector">12.2. Salesforce Connector</a> +<ul class="sectlevel3"> +<li><a href="#_getting_started_2">12.2.1. Getting started</a></li> +<li><a href="#_properties">12.2.2. Properties</a></li> +<li><a href="#_hot_deploying_updates_to_the_salesforce_connector_for_developers">12.2.3. Hot-deploying updates to the Salesforce connector (for developers)</a></li> +<li><a href="#_using_the_salesforce_workbench_for_testing_rest_api">12.2.4. Using the Salesforce Workbench for testing REST API</a></li> +<li><a href="#_setting_up_streaming_push_queries">12.2.5. Setting up Streaming Push queries</a></li> +<li><a href="#_executing_the_unit_tests">12.2.6. Executing the unit tests</a></li> +</ul> +</li> +<li><a href="#_mailchimp_connector">12.3. MailChimp Connector</a> +<ul class="sectlevel3"> +<li><a href="#_getting_started_3">12.3.1. Getting started</a></li> +</ul> +</li> +</ul> +</li> +<li><a href="#_developers">13. Developers</a> +<ul class="sectlevel2"> +<li><a href="#_building">13.1. Building</a> +<ul class="sectlevel3"> +<li><a href="#_initial_setup">13.1.1. Initial Setup</a></li> +<li><a href="#_building_2">13.1.2. Building</a></li> +<li><a href="#_installing_an_elasticsearch_server">13.1.3. Installing an ElasticSearch server</a></li> +<li><a href="#_deploying_the_generated_binary_package">13.1.4. Deploying the generated binary package</a></li> +<li><a href="#_deploying_into_an_existing_karaf_server">13.1.5. Deploying into an existing Karaf server</a></li> +<li><a href="#_jdk_selection_on_mac_os_x">13.1.6. JDK Selection on Mac OS X</a></li> +<li><a href="#_running_the_integration_tests">13.1.7. Running the integration tests</a></li> +<li><a href="#_testing_with_an_example_page">13.1.8. Testing with an example page</a></li> +</ul> +</li> +<li><a href="#_ssh_shell_commands">13.2. SSH Shell Commands</a> +<ul class="sectlevel3"> +<li><a href="#_using_the_shell">13.2.1. Using the shell</a></li> +<li><a href="#_lifecycle_commands">13.2.2. Lifecycle commands</a></li> +<li><a href="#_runtime_commands">13.2.3. Runtime commands</a></li> +</ul> +</li> +<li><a href="#_writing_plugins">13.3. Writing Plugins</a></li> +<li><a href="#_types_vs_instances">13.4. Types vs. instances</a></li> +<li><a href="#_plugin_structure">13.5. Plugin structure</a></li> +<li><a href="#_extension_points">13.6. Extension points</a> +<ul class="sectlevel3"> +<li><a href="#_actiontype">13.6.1. ActionType</a></li> +<li><a href="#_conditiontype">13.6.2. ConditionType</a></li> +<li><a href="#_persona_2">13.6.3. Persona</a></li> +<li><a href="#_propertymergestrategytype">13.6.4. PropertyMergeStrategyType</a></li> +<li><a href="#_propertytype">13.6.5. PropertyType</a></li> +<li><a href="#_rule_2">13.6.6. Rule</a></li> +<li><a href="#_scoring">13.6.7. Scoring</a></li> +<li><a href="#_segments">13.6.8. Segments</a></li> +<li><a href="#_tag">13.6.9. Tag</a></li> +<li><a href="#_valuetype">13.6.10. ValueType</a></li> +</ul> +</li> +<li><a href="#_custom_plugins">13.7. Custom plugins</a> +<ul class="sectlevel3"> +<li><a href="#_creating_a_plugin">13.7.1. Creating a plugin</a></li> +<li><a href="#_deployment_and_custom_definition">13.7.2. Deployment and custom definition</a></li> +<li><a href="#_predefined_segments">13.7.3. Predefined segments</a></li> +<li><a href="#_predefined_rules">13.7.4. Predefined rules</a></li> +<li><a href="#_predefined_properties">13.7.5. Predefined properties</a></li> +<li><a href="#_predefined_child_conditions">13.7.6. Predefined child conditions</a></li> +<li><a href="#_predefined_personas">13.7.7. Predefined personas</a></li> +<li><a href="#_custom_action_types">13.7.8. Custom action types</a></li> +<li><a href="#_custom_condition_types">13.7.9. Custom condition types</a></li> +</ul> +</li> +<li><a href="#_migration_patches">13.8. Migration patches</a></li> +</ul> +</li> +</ul> +</div> +</div> +<div id="content"> +<div id="preamble"> +<div class="sectionbody"> +<div class="imageblock text-center"> +<div class="content"> +<img src="images/asf_logo_url.png" alt="asf logo url"> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="_quick_start">1. Quick start</h2> +<div class="sectionbody"> +<div class="sect2"> +<h3 id="_five_minutes_quickstart">1.1. Five Minutes QuickStart</h3> +<div class="paragraph"> +<p>1) Install JDK 8 (<a href="https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html"; class="bare">https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html</a>) and make sure you set the +JAVA_HOME variable <a href="https://docs.oracle.com/cd/E19182-01/820-7851/inst_cli_jdk_javahome_t/"; class="bare">https://docs.oracle.com/cd/E19182-01/820-7851/inst_cli_jdk_javahome_t/</a> (see our <a href="#_jdk_compatibility">Getting Started</a> guide for more information on JDK compatibility)</p> +</div> +<div class="paragraph"> +<p>2) Download ElasticSearch here : <a href="https://www.elastic.co/downloads/past-releases/elasticsearch-7-4-2"; class="bare">https://www.elastic.co/downloads/past-releases/elasticsearch-7-4-2</a> (please <strong>make sure</strong> you use the proper version : 7.4.2)</p> +</div> +<div class="paragraph"> +<p>3) Uncompress it and change the <code>config/elasticsearch.yml</code> to include the following config : <code>cluster.name: contextElasticSearch</code></p> +</div> +<div class="paragraph"> +<p>4) Launch ElasticSearch using : <code>bin/elasticsearch</code></p> +</div> +<div class="paragraph"> +<p>5) Download Apache Unomi here : <a href="https://unomi.apache.org/download.html"; class="bare">https://unomi.apache.org/download.html</a></p> +</div> +<div class="paragraph"> +<p>6) Start it using : <code>./bin/karaf</code></p> +</div> +<div class="paragraph"> +<p>7) Start the Apache Unomi packages using <code>unomi:start</code> in the Apache Karaf Shell</p> +</div> +<div class="paragraph"> +<p>8) Wait for startup to complete</p> +</div> +<div class="paragraph"> +<p>9) Try accessing <a href="https://localhost:9443/cxs/cluster"; class="bare">https://localhost:9443/cxs/cluster</a> with username/password: <code>karaf/karaf</code> . You might get a certificate warning in your browser, just accept it despite the warning it is safe.</p> +</div> +<div class="paragraph"> +<p>10) Request your first context by simply accessing : <a href="http://localhost:8181/cxs/context.js?sessionId=1234"; class="bare">http://localhost:8181/cxs/context.js?sessionId=1234</a></p> +</div> +<div class="paragraph"> +<p>11) If something goes wrong, you should check the logs in <code>./data/log/karaf.log</code>. If you get errors on ElasticSearch, +make sure you are using the proper version.</p> +</div> +<div class="paragraph"> +<p>Next steps:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>Connect to <a href="http://localhost:8181"; class="bare">http://localhost:8181</a> to try our some live examples (such as the web tracker)</p> +</li> +<li> +<p>Trying our integration <a href="#_samples">samples page</a></p> +</li> +<li> +<p>Learning more about the <a href="#_web_tracker">web tracker</a></p> +</li> +</ul> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="_first_steps_with_apache_unomi">2. First steps with Apache Unomi</h2> +<div class="sectionbody"> +<div class="sect2"> +<h3 id="_getting_started_with_unomi">2.1. Getting started with Unomi</h3> +<div class="paragraph"> +<p>We will first get you up and running with an example. We will then lift the corner of the cover somewhat and explain +in greater details what just happened.</p> +</div> +<div class="sect3"> +<h4 id="_prerequisites">2.1.1. Prerequisites</h4> +<div class="paragraph"> +<p>This document assumes working knowledge of <a href="https://git-scm.com/";>git</a> to be able to retrieve the code for Unomi and the example. +Additionally, you will require a working Java 8 or above install. Refer to <a href="http://www.oracle.com/technetwork/java/javase/";>http://www.oracle.com/technetwork/java/javase/</a> for details on how to download and install Java SE 8 or greater.</p> +</div> +<div class="sect4"> +<h5 id="_jdk_compatibility">JDK compatibility</h5> +<div class="paragraph"> +<p>Starting with Java 9, Oracle made some big changes to the Java platform releases. This is why Apache Unomi is focused on +supporting the Long Term Supported versions of the JDK, currently versions 8 and 11. We do not test with intermediate +versions so they may or may not work properly. Currently the most tested version is version 8 and version 11 is also +supported.</p> +</div> +<div class="paragraph"> +<p>Also, as there are new licensing restrictions on JDKs provided by Oracle for production usages, Apache Unomi has also +added support for OpenJDK builds. Other JDK distributions might also work but are not regularly tested so you should use +them at your own risks.</p> +</div> +</div> +<div class="sect4"> +<h5 id="_elasticsearch_compatibility">ElasticSearch compatibility</h5> +<div class="paragraph"> +<p>Starting with version 1.5.0 Apache Unomi adds compatibility with ElasticSearch 7.4 . It is highly recommended to use the +ElasticSearch version provided by the documentation when possible. However minor versions (7.4.x) should also work, and +one version higher (7.5) will usually work. Going higher than that is risky given the way that ElasticSearch is developed +and breaking changes are introduced quite often. If in doubt, don’t hesitate to check with the Apache Unomi community +to get the latest information about ElasticSearch version compatibility.</p> +</div> +</div> +</div> +<div class="sect3"> +<h4 id="_running_unomi">2.1.2. Running Unomi</h4> +<div class="sect4"> +<h5 id="_start_unomi">Start Unomi</h5> +<div class="paragraph"> +<p>Start Unomi according to the <a href="#_five_minutes_quickstart">five minutes quick start</a> or by compiling using the +<a href="#_building">building instructions</a>. Once you have Karaf running, + you should wait until you see the following messages on the Karaf console:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>Initializing user list service endpoint... +Initializing geonames service endpoint... +Initializing segment service endpoint... +Initializing scoring service endpoint... +Initializing campaigns service endpoint... +Initializing rule service endpoint... +Initializing profile service endpoint... +Initializing cluster service endpoint...</code></pre> +</div> +</div> +<div class="paragraph"> +<p>This indicates that all the Unomi services are started and ready to react to requests. You can then open a browser and go to <code><a href="http://localhost:8181/cxs"; class="bare">http://localhost:8181/cxs</a></code> to see the list of +available RESTful services or retrieve an initial context at <code><a href="http://localhost:8181/cxs/context.json"; class="bare">http://localhost:8181/cxs/context.json</a></code> (which isn’t very useful at this point).</p> +</div> +<div class="paragraph"> +<p>You can now find an introduction page at the following location: <a href="http://localhost:8181"; class="bare">http://localhost:8181</a></p> +</div> +<div class="paragraph"> +<p>Also now that your service is up and running you can go look at the +<a href="#_request_examples">request examples</a> to learn basic +requests you can do once your server is up and running.</p> +</div> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="_recipes">2.2. Recipes</h3> +<div class="sect3"> +<h4 id="_introduction">2.2.1. Introduction</h4> +<div class="paragraph"> +<p>In this section of the documentation we provide quick recipes focused on helping you achieve a specific result with +Apache Unomi.</p> +</div> +</div> +<div class="sect3"> +<h4 id="_how_to_read_a_profile">2.2.2. How to read a profile</h4> +<div class="paragraph"> +<p>The simplest way to retrieve profile data for the current profile is to simply send a request to the /cxs/context.json +endpoint. However you will need to send a body along with that request. Here’s an example:</p> +</div> +<div class="paragraph"> +<p>Here is an example that will retrieve all the session and profile properties, as well as the profile’s segments and scores</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>curl -X POST http://localhost:8181/cxs/context.json?sessionId=1234 \ +-H "Content-Type: application/json" \ +-d @- <<'EOF' +{ + "source": { + "itemId":"homepage", + "itemType":"page", + "scope":"example" + }, + "requiredProfileProperties":["*"], + "requiredSessionProperties":["*"], + "requireSegments":true, + "requireScores":true +} +EOF</code></pre> +</div> +</div> +<div class="paragraph"> +<p>The <code>requiredProfileProperties</code> and <code>requiredSessionProperties</code> are properties that take an array of property names +that should be retrieved. In this case we use the wildcard character '*' to say we want to retrieve all the available +properties. The structure of the JSON object that you should send is a JSON-serialized version of the <a href="http://unomi.apache.org/unomi-api/apidocs/org/apache/unomi/api/ContextRequest.html";>ContextRequest</a> +Java class.</p> +</div> +<div class="paragraph"> +<p>Note that it is also possible to access a profile’s data through the /cxs/profiles/ endpoint but that really should be +reserved to administrative purposes. All public accesses should always use the /cxs/context.json endpoint for consistency +and security.</p> +</div> +</div> +<div class="sect3"> +<h4 id="_how_to_update_a_profile_from_the_public_internet">2.2.3. How to update a profile from the public internet</h4> +<div class="paragraph"> +<p>Before we get into how to update a profile directly from a request coming from the public internet, we’ll quickly talk +first about how NOT to do it, because we often see users using the following anti-patterns.</p> +</div> +<div class="sect4"> +<h5 id="_how_not_to_update_a_profile_from_the_public_internet">How NOT to update a profile from the public internet</h5> +<div class="paragraph"> +<p>Please avoid using the /cxs/profile endpoint. This endpoint was initially the only way to update a profile but it has +multiple issues:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>it requires authenticated access. The temptation can be great to use this endpoint because it is simple to access +but the risk is that developers might include the credentials to access it in non-secure parts of code such as +client-side code. Since there is no difference between this endpoint and any other administration-focused endpoints, +attackers could easily re-use stolen credentials to wreak havock on the whole platform.</p> +</li> +<li> +<p>No history of profile modifications is kept: this can be a problem for multiple reasons: you might want to keep an +trail of profile modifications, or even a history of profile values in case you want to understand how a profile +property was modified.</p> +</li> +<li> +<p>Even when protected using some kind of proxy, potentially the whole profile properties might be modified, including +ones that you might not want to be overriden.</p> +</li> +</ul> +</div> +</div> +<div class="sect4"> +<h5 id="_recommended_ways_to_update_a_profile">Recommended ways to update a profile</h5> +<div class="paragraph"> +<p>Instead you can use the following solutions to update profiles:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>(Preferred) Use you own custom event(s) to send data you want to be inserted in a profile, and use rules to map the +event data to the profile. This is simpler than it sounds, as usually all it requires is setting up a simple rule and +you’re ready to update profiles using events. This is also the safest way to update a profile because if you design your +events to be as specific as possible to your needs, only the data that you specified will be copied to the profile, +making sure that even in the case an attacker tries to send more data using your custom event it will simply be ignored.</p> +</li> +<li> +<p>Use the protected built-in "updateProperties" event. This event is designed to be used for administrative purposes +only. Again, prefer the custom events solution because as this is a protected event it will require sending the Unomi +key as a request header, and as Unomi only supports a single key for the moment it could be problematic if the key is +intercepted. But at least by using an event you will get the benefits of auditing and historical property modification +tracing.</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>Let’s go into more detail about the preferred way to update a profile. Let’s consider the following example of a rule:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>curl -X POST http://localhost:8181/cxs/rules \ +--user karaf:karaf \ +-H "Content-Type: application/json" \ +-d @- <<'EOF' +{ + "metadata": { + "id": "setContactInfo", + "name": "Copy the received contact info to the current profile", + "description": "Copies the contact info received in a custom event called 'contactInfoSubmitted' to the current profile" + }, + "raiseEventOnlyOnceForSession": false, + "condition": { + "type": "eventTypeCondition", + "parameterValues": { + "eventTypeId": "contactInfoSubmitted" + } + }, + "actions": [ + { + "type": "setPropertyAction", + "parameterValues": { + "setPropertyName": "properties(firstName)", + "setPropertyValue": "eventProperty::properties(firstName)", + "setPropertyStrategy": "alwaysSet" + } + }, + { + "type": "setPropertyAction", + "parameterValues": { + "setPropertyName": "properties(lastName)", + "setPropertyValue": "eventProperty::properties(lastName)", + "setPropertyStrategy": "alwaysSet" + } + }, + { + "type": "setPropertyAction", + "parameterValues": { + "setPropertyName": "properties(email)", + "setPropertyValue": "eventProperty::properties(email)", + "setPropertyStrategy": "alwaysSet" + } + } + ] +} +EOF</code></pre> +</div> +</div> +<div class="paragraph"> +<p>What this rule does is that it listen for a custom event (events don’t need any registration, you can simply start +sending them to Apache Unomi whenever you like) of type 'contactInfoSubmitted' and it will search for properties called +'firstName', 'lastName' and 'email' and copy them over to the profile with corresponding property names. You could of +course change any of the property names to find your needs. For example you might want to prefix the profile properties +with the source of the event, such as 'mobileApp:firstName'.</p> +</div> +<div class="paragraph"> +<p>You could then simply send the <code>contactInfoSubmitted</code> event using a request similar to this one:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>curl -X POST http://localhost:8181/cxs/eventcollector \ +-H "Content-Type: application/json" \ +-d @- <<'EOF' +{ + "sessionId" : "1234", + "events":[ + { + "eventType":"contactInfoSubmitted", + "scope": "example", + "source":{ + "itemType": "site", + "scope":"example", + "itemId": "mysite" + }, + "target":{ + "itemType":"form", + "scope":"example", + "itemId":"contactForm" + }, + "properties" : { + "firstName" : "John", + "lastName" : "Doe", + "email" : "[email protected]" + } + } + ] +} +EOF</code></pre> +</div> +</div> +</div> +</div> +<div class="sect3"> +<h4 id="_how_to_search_for_profile_events">2.2.4. How to search for profile events</h4> +<div class="paragraph"> +<p>Sometimes you want to retrieve events for a known profile. You will need to provide a query in the body of the request +that looks something like this (and <a href="https://unomi.apache.org/rest-api-doc/#1768188821";>documentation is available in the REST API</a>) :</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>curl -X POST http://localhost:8181/cxs/events/search \ +--user karaf:karaf \ +-H "Content-Type: application/json" \ +-d @- <<'EOF' +{ "offset" : 0, + "limit" : 20, + "condition" : { + "type": "eventPropertyCondition", + "parameterValues" : { + "propertyName" : "profileId", + "comparisonOperator" : "equals", + "propertyValue" : "PROFILE_ID" + } + } +} +EOF</code></pre> +</div> +</div> +<div class="paragraph"> +<p>where PROFILE_ID is a profile identifier. This will indeed retrieve all the events for a given profile.</p> +</div> +</div> +<div class="sect3"> +<h4 id="_how_to_create_a_new_rule">2.2.5. How to create a new rule</h4> +<div class="paragraph"> +<p>There are basically two ways to create a new rule :</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>Using the REST API</p> +</li> +<li> +<p>Packaging it as a predefined rule in a plugin</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>In both cases the JSON structure for the rule will be exactly the same, and in most scenarios it will be more +interesting to use the REST API to create and manipulate rules, as they don’t require any development or deployments +on the Apache Unomi server.</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>curl -X POST http://localhost:8181/cxs/rules \ +--user karaf:karaf \ +-H "Content-Type: application/json" \ +-d @- <<'EOF' +{ + "metadata": { + "id": "exampleEventCopy", + "name": "Example Copy Event to Profile", + "description": "Copy event properties to profile properties" + }, + "condition": { + "type": "eventTypeCondition", + "parameterValues": { + "eventTypeId" : "myEvent" + } + }, + "actions": [ + { + "parameterValues": { + }, + "type": "allEventToProfilePropertiesAction" + } + ] +} +EOF</code></pre> +</div> +</div> +<div class="paragraph"> +<p>The above rule will be executed if the incoming event is of type <code>myEvent</code> and will simply copy all the properties +contained in the event to the current profile.</p> +</div> +</div> +<div class="sect3"> +<h4 id="_how_to_search_for_profiles">2.2.6. How to search for profiles</h4> +<div class="paragraph"> +<p>In order to search for profiles you will have to use the /cxs/profiles/search endpoint that requires a Query JSON +structure. Here’s an example of a profile search with a Query object:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>curl -X POST http://localhost:8181/cxs/profiles/search \ +--user karaf:karaf \ +-H "Content-Type: application/json" \ +-d @- <<'EOF' +{ + "text" : "unomi", + "offset" : 0, + "limit" : 10, + "sortby" : "properties.lastName:asc,properties.firstName:desc", + "condition" : { + "type" : "booleanCondition", + "parameterValues" : { + "operator" : "and", + "subConditions" : [ + { + "type": "profilePropertyCondition", + "parameterValues": { + "propertyName": "properties.leadAssignedTo", + "comparisonOperator": "exists" + } + }, + { + "type": "profilePropertyCondition", + "parameterValues": { + "propertyName": "properties.lastName", + "comparisonOperator": "exists" + } + } + ] + } + } +} +EOF</code></pre> +</div> +</div> +<div class="paragraph"> +<p>In the above example, you search for all the profiles that have the <code>leadAssignedTo</code> and <code>lastName</code> properties and that +have the <code>unomi</code> value anywhere in their profile property values. You are also specifying that you only want 10 results +beginning at offset 0. The results will be also sorted in alphabetical order for the <code>lastName</code> property value, and then +by reverse alphabetical order for the <code>firstName</code> property value.</p> +</div> +<div class="paragraph"> +<p>As you can see, queries can be quite complex. Please remember that the more complex the more resources it will consume +on the server and potentially this could affect performance.</p> +</div> +</div> +<div class="sect3"> +<h4 id="_getting_updating_consents">2.2.7. Getting / updating consents</h4> +<div class="paragraph"> +<p>You can find information on how to retrieve or create/update consents in the <a href="#_consent_api">Consent API</a> section.</p> +</div> +</div> +<div class="sect3"> +<h4 id="_how_to_send_a_login_event_to_unomi">2.2.8. How to send a login event to Unomi</h4> +<div class="paragraph"> +<p>Tracking logins must be done carefully with Unomi. A login event is considered a "privileged" event and therefore for +not be initiated from the public internet. Ideally user authentication should always be validated by a trusted third- +party even if it is a well-known social platform such as Facebook or Twitter. Basically what should NEVER be done:</p> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>Login to a social platform</p> +</li> +<li> +<p>Call back to the originating page</p> +</li> +<li> +<p>Send a login event to Unomi from the page originating the login in step 1</p> +</li> +</ol> +</div> +<div class="paragraph"> +<p>The problem with this, is that any attacker could simply directly call step 3 without any kind of security. Instead the +flow should look something like this:</p> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>Login to a social platform</p> +</li> +<li> +<p>Call back to a special secured system that performs an server-to-server call to send the login event to Apache +Unomi using the Unomi key.</p> +</li> +</ol> +</div> +<div class="paragraph"> +<p>For simplicity reasons, in our login example, the first method is used, but it really should never be done like this +in production because of the aforementioned security issues. The second method, although a little more involved, is +much preferred.</p> +</div> +<div class="paragraph"> +<p>When sending a login event, you can setup a rule that can check a profile property to see if profiles can be merged on an +universal identifier such as an email address.</p> +</div> +<div class="paragraph"> +<p>In our login sample we provide an example of such a rule. You can find it here:</p> +</div> +<div class="paragraph"> +<p><a href="https://github.com/apache/unomi/blob/master/samples/login-integration/src/main/resources/META-INF/cxs/rules/exampleLogin.json"; class="bare">https://github.com/apache/unomi/blob/master/samples/login-integration/src/main/resources/META-INF/cxs/rules/exampleLogin.json</a></p> +</div> +<div class="paragraph"> +<p>As you can see in this rule, we call an action called :</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>mergeProfilesOnPropertyAction</pre> +</div> +</div> +<div class="paragraph"> +<p>with as a parameter value the name of the property on which to perform the merge (the email). What this means is that +upon successful login using an email, Unomi will look for other profiles that have the same email and merge them into +a single profile. Because of the merge, this should only be done for authenticated profiles, otherwise this could be a +security issue since it could be a way to load data from other profiles by merging their data !</p> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="_request_examples">2.3. Request examples</h3> +<div class="sect3"> +<h4 id="_retrieving_your_first_context">2.3.1. Retrieving your first context</h4> +<div class="paragraph"> +<p>You can retrieve a context using curl like this :</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>curl http://localhost:8181/cxs/context.js?sessionId=1234</code></pre> +</div> +</div> +<div class="paragraph"> +<p>This will retrieve a JavaScript script that contains a <code>cxs</code> object that contains the context with the current user +profile, segments, scores as well as functions that makes it easier to perform further requests (such as collecting +events using the cxs.collectEvents() function).</p> +</div> +</div> +<div class="sect3"> +<h4 id="_retrieving_a_context_as_a_json_object">2.3.2. Retrieving a context as a JSON object.</h4> +<div class="paragraph"> +<p>If you prefer to retrieve a pure JSON object, you can simply use a request formed like this:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>curl http://localhost:8181/cxs/context.json?sessionId=1234</code></pre> +</div> +</div> +</div> +<div class="sect3"> +<h4 id="_accessing_profile_properties_in_a_context">2.3.3. Accessing profile properties in a context</h4> +<div class="paragraph"> +<p>By default, in order to optimize the amount of data sent over the network, Apache Unomi will not send the content of +the profile or session properties. If you need this data, you must send a JSON object to configure the resulting output +of the context.js(on) servlet.</p> +</div> +<div class="paragraph"> +<p>Here is an example that will retrieve all the session and profile properties, as well as the profile’s segments and +scores</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>curl -X POST http://localhost:8181/cxs/context.json?sessionId=1234 \ +-H "Content-Type: application/json" \ +-d @- <<'EOF' +{ + "source": { + "itemId":"homepage", + "itemType":"page", + "scope":"example" + }, + "requiredProfileProperties":["*"], + "requiredSessionProperties":["*"], + "requireSegments":true, + "requireScores":true +} +EOF</code></pre> +</div> +</div> +<div class="paragraph"> +<p>The <code>requiredProfileProperties</code> and <code>requiredSessionProperties</code> are properties that take an array of property names +that should be retrieved. In this case we use the wildcard character '*' to say we want to retrieve all the available +properties. The structure of the JSON object that you should send is a JSON-serialized version of the <a href="http://unomi.apache.org/unomi-api/apidocs/org/apache/unomi/api/ContextRequest.html";>ContextRequest</a> +Java class.</p> +</div> +</div> +<div class="sect3"> +<h4 id="_sending_events_using_the_context_servlet">2.3.4. Sending events using the context servlet</h4> +<div class="paragraph"> +<p>At the same time as you are retrieving the context, you can also directly send events in the ContextRequest object as +illustrated in the following example:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>curl -X POST http://localhost:8181/cxs/context.json?sessionId=1234 \ +-H "Content-Type: application/json" \ +-d @- <<'EOF' +{ + "source":{ + "itemId":"homepage", + "itemType":"page", + "scope":"example" + }, + "events":[ + { + "eventType":"view", + "scope": "example", + "source":{ + "itemType": "site", + "scope":"example", + "itemId": "mysite" + }, + "target":{ + "itemType":"page", + "scope":"example", + "itemId":"homepage", + "properties":{ + "pageInfo":{ + "referringURL":"" + } + } + } + } + ] +} +EOF</code></pre> +</div> +</div> +<div class="paragraph"> +<p>Upon received events, Apache Unomi will execute all the rules that match the current context, and return an updated context. +This way of sending events is usually used upon first loading of a page. If you want to send events after the page has +finished loading you could either do a second call and get an updating context, or if you don’t need the context and want +to send events in a network optimal way you can use the eventcollector servlet (see below).</p> +</div> +</div> +<div class="sect3"> +<h4 id="_sending_events_using_the_eventcollector_servlet">2.3.5. Sending events using the eventcollector servlet</h4> +<div class="paragraph"> +<p>If you only need to send events without retrieving a context, you should use the eventcollector servlet that is optimized +respond quickly and minimize network traffic. Here is an example of using this servlet:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>curl -X POST http://localhost:8181/cxs/eventcollector \ +-H "Content-Type: application/json" \ +-d @- <<'EOF' +{ + "sessionId" : "1234", + "events":[ + { + "eventType":"view", + "scope": "example", + "source":{ + "itemType": "site", + "scope":"example", + "itemId": "mysite" + }, + "target":{ + "itemType":"page", + "scope":"example", + "itemId":"homepage", + "properties":{ + "pageInfo":{ + "referringURL":"" + } + } + } + } + ] +} +EOF</code></pre> +</div> +</div> +<div class="paragraph"> +<p>Note that the eventcollector executes the rules but does not return a context. If is generally used after a page is loaded +to send additional events.</p> +</div> +</div> +<div class="sect3"> +<h4 id="_where_to_go_from_here">2.3.6. Where to go from here</h4> +<div class="ulist"> +<ul> +<li> +<p>You can find more <a href="#_useful_apache_unomi_urls">useful Apache Unomi URLs</a> that can be used in the same way as the above examples.</p> +</li> +<li> +<p>You may want to know integrate the provided <a href="#_web_tracker">web tracker</a> into your web site.</p> +</li> +<li> +<p>Read the <a href="#_twitter_sample">Twitter sample</a> documentation that contains a detailed example of how to integrate with Apache Unomi.</p> +</li> +</ul> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="_web_tracker">2.4. Web Tracker</h3> +<div class="paragraph"> +<p>This extension is providing the web tracker to start collecting visitors data on your website. +The tracker is implemented as an integration of <a href="https://github.com/segmentio/analytics.js";>analytics.js</a> for Unomi.</p> +</div> +<div class="sect3"> +<h4 id="_getting_started">2.4.1. Getting started</h4> +<div class="paragraph"> +<p>Extension can be tested at : <code><a href="http://localhost:8181/tracker/index.html"; class="bare">http://localhost:8181/tracker/index.html</a></code></p> +</div> +<div class="paragraph"> +<p>In your page include unomiOptions and include code snippet from <code>snippet.min.js</code> :</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code><script type="text/javascript"> + var unomiOption = { + scope: 'realEstateManager', + url: 'http://localhost:8181' + }; + window.unomiTracker||(window.unomiTracker={}),function(){function e(e){for(unomiTracker.initialize({"Apache Unomi":unomiOption});n.length>0;){var r=n.shift(),t=r.shift();unomiTracker[t]&&unomiTracker[t].apply(unomiTracker,r)}}for(var n=[],r=["trackSubmit","trackClick","trackLink","trackForm","initialize","pageview","identify","reset","group","track","ready","alias","debug","page","once","off","on","personalize"],t=0;t<r.length;t++){var i=r[t];window.unomiTracker[i]=function(e){return function(){var r=Array.prototype.slice.call(arguments);return r.unshift(e),n.push(r),window.unomiTracker}}(i)}unomiTracker.load=function(){var n=document.createElement("script");n.type="text/javascript",n.async=!0,n.src=unomiOption.url+"/tracker/unomi-tracker.min.js",n.addEventListener?n.addEventListener("load",function(n){"function"==typeof e&&e(n)},!1):n.onreadystatechange=function(){"complete"!==this.readyState&&"loaded"!==this.readyState||e(window.event)};var r= document.getElementsByTagName("script")[0];r.parentNode.insertBefore(n,r)},document.addEventListener("DOMContentLoaded",unomiTracker.load),unomiTracker.page()}(); +</script></code></pre> +</div> +</div> +<div class="paragraph"> +<p><code>window.unomiTracker</code> can be used to send additional events when needed.</p> +</div> +<div class="paragraph"> +<p>Check analytics.js API <a href="https://segment.com/docs/sources/website/analytics.js/";>here</a>. +All methods can be used on <code>unomiTracker</code> object, although not all event types are supported by Unomi intergation.</p> +</div> +</div> +<div class="sect3"> +<h4 id="_how_to_contribute">2.4.2. How to contribute</h4> +<div class="paragraph"> +<p>The source code is in the folder javascript with a package.json, the file to update is <code>analytics.js-integration-apache-unomi.js</code> apply your modification in this file then use the command <code>yarn build</code> to compile a new JS file. +Then you can use the test page to try your changes <code><a href="http://localhost:8181/tracker/index.html"; class="bare">http://localhost:8181/tracker/index.html</a></code>.</p> +</div> +</div> +<div class="sect3"> +<h4 id="_tracking_page_views">2.4.3. Tracking page views</h4> +<div class="paragraph"> +<p>In the initialize call, the tracker will generate an implicit page view event, which by default will be populated with +the following information:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code class="language-javascript" data-lang="javascript"> window.digitalData.page = window.digitalData.page || { + path: location.pathname + location.hash, + pageInfo: { + pageName: document.title, + pageID : location.pathname + location.hash, + pagePath : location.pathname + location.hash, + destinationURL: location.href + } + }</code></pre> +</div> +</div> +<div class="paragraph"> +<p>Now if you want to provide your own custom page information for the initial page view, you can simply do it like this:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code class="language-`javascript" data-lang="`javascript"> unomiTracker.initialize({ + scope: 'myScope', + url: 'http://unomi:8181', // we use an empty URL to make it relative to this page. + initialPageProperties: { + path: path, + pageInfo: { + destinationURL: location.href, + tags: ["tag1", "tag2", "tag3"], + categories: ["category1", "category2", "category3"] + }, + interests: { + "interest1": 1, + "interest2": 2, + "interest3": 3 + } + } + });</code></pre> +</div> +</div> +<div class="paragraph"> +<p>`</p> +</div> +<div class="paragraph"> +<p>Also note that the FIRST call to unomiTracker.page() will be IGNORED because of this initial page view.This is the +way that the Analytics.js library handles it.So make sure you are aware of this when calling it.This is to avoid having +two page views on a single call and to be compatible with old versions that did use the explicit call.</p> +</div> +<div class="paragraph"> +<p>By default the script will track page views, but maybe you want to take control over this mechanism of add page views +to a single page application.In order to generate a page view programmatically from Javascript you can use code similar +to this :</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code> <script type="text/javascript"> + // This is an example of how to provide more details page properties to the view event. This can be useful + // in the case of an SPA that wants to provide information about a view that has metadata such as categories, + // tags or interests. + path = location.pathname + location.hash; + properties = { + path: path, + pageInfo: { + destinationURL: location.href, + tags : [ "tag1", "tag2", "tag3"], + categories : ["category1", "category2", "category3"], + }, + interests : { + "interest1" : 1, + "interest2" : 2, + "interest3" : 3 + } + }; + console.log(properties); + // this will trigger a second page view for the same page (the first page view is in the tracker snippet). + window.unomiTracker.page(properties); + </script></code></pre> +</div> +</div> +<div class="paragraph"> +<p>Here is a more detail view of what you may include in the pageInfo object :</p> +</div> +<table class="tableblock frame-all grid-all stretch"> +<caption class="title">Table 1. PageInfo Properties</caption> +<colgroup> +<col style="width: 50%;"> +<col style="width: 50%;"> +</colgroup> +<thead> +<tr> +<th class="tableblock halign-left valign-top">Name</th> +<th class="tableblock halign-left valign-top">Description</th> +</tr> +</thead> +<tbody> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">pageID</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">A unique identifier in string format for the page. Default value : page path</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">pageName</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">A user-displayed name for the page. Default value : page title</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">pagePath</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">The path of the page, stored by Unomi. This value should be the same as the one passed in the <code>page</code> property of the +object passed to the unomiTracker call. Default value : page path</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">destinationURL</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">The full URL for the page view. This doesn’t have to be a real existing URL it could be an internal SPA route. Default value : page URL</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">referringURL</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">The referringURL also known as the previous URL of the page/screen viewed. Default value : page referrer URL</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">tags</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">A String array of tag identifiers. For example <code>['tag1', 'tag2', 'tag3']</code></p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">categories</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">A String array of category identifiers. For example <code>['category1', 'category2', 'category3']</code></p></td> +</tr> +</tbody> +</table> +<div class="paragraph"> +<p>The <code>interests</code> object is basically list of interests with "weights" attached to them.These interests will be accumulated +in Apache Unomi on profiles to indicate growing interest over time for specific topics.These are freely defined and +will be accepted by Apache Unomi without needing to declare them previously anywhere (the same is true for tags and +categories).</p> +</div> +</div> +<div class="sect3"> +<h4 id="_tracking_form_submissions">2.4.4. Tracking form submissions</h4> +<div class="paragraph"> +<p>Using the web tracker you can also track form submissions. In order to do this a few steps are required to get a form’s +submission to be tracked and then its form values to be sent as events to Apache Unomi. Finally setting up a rule to +react to the incoming event will help use the form values to perform any action that is desired.</p> +</div> +<div class="paragraph"> +<p>Let’s look at a concrete example. Before we get started you should know that this example is already available to +directly test in Apache Unomi at the following URL :</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>http://localhost:8181/tracker</pre> +</div> +</div> +<div class="paragraph"> +<p>Simply modify the form values and click submit and it will perform all the steps we are describing below.</p> +</div> +<div class="paragraph"> +<p>So here is the form we want to track :</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code><form id="testFormTracking" action="#" name="testFormTracking"> + <label for="firstName">First name</label> + <input type="text" id="firstName" name="firstName" value="John"/> + + <label for="lastName">Last name</label> + <input type="text" id="lastName" name="lastName" value="Doe"/> + + <label for="email">Email</label> + <input type="email" id="email" name="email" value="[email protected]"/> + + <input type="submit" name="submitButton" value="Submit"/> +</form></code></pre> +</div> +</div> +<div class="paragraph"> +<p>As you can see it’s composed of three fields - firstName, lastName and email - as well as a submit button. In order to +track it we can add directly under the following snippet :</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code><script type="text/javascript"> + window.addEventListener("load", function () { + var form = document.getElementById('testFormTracking'); + unomiTracker.trackForm(form, 'formSubmitted', {formName: form.name}); + }); +</script></code></pre> +</div> +</div> +<div class="paragraph"> +<p>What this snippet does is retrieve the form using its element ID and then uses the unomiTracker to track form submissions. +Be careful to always use in the form event name a string that starts with <code>form</code> in order for the event to be sent back +to Unomi. Also the form name is also a mandatory parameter that will be passed to Unomi inside a event of type <code>form</code> under +the <code>target.itemId</code> property name.</p> +</div> +<div class="paragraph"> +<p>Here is an example of the event that gets sent back to Apache Unomi:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>{ + "itemId" : "cd627012-963e-4bb5-97f0-480990b41254", + "itemType" : "event", + "scope" : "realEstateManager", + "version" : 1, + "eventType" : "form", + "sessionId" : "aaad09aa-88c2-67bd-b106-5a47ded43ead", + "profileId" : "48563fd0-6319-4260-8dba-ae421beba26f", + "timeStamp" : "2018-11-23T16:32:26Z", + "properties" : { + "firstName" : "John", + "lastName" : "Doe", + "email" : "[email protected]", + "submitButton" : "Submit" + }, + "source" : { + "itemId" : "/tracker/", + "itemType" : "page", + "scope" : "realEstateManager", + "version" : null, + "properties" : { + "pageInfo" : { + "destinationURL" : "http://localhost:8181/tracker/?firstName=Bill&lastName=Gates&email=bgates%40microsoft.com";, + "pageID" : "/tracker/", + "pagePath" : "/tracker/", + "pageName" : "Apache Unomi Web Tracker Test Page", + "referringURL" : "http://localhost:8181/tracker/?firstName=John&lastName=Doe&email=johndoe%40acme.com"; + }, + "attributes" : [ ], + "consentTypes" : [ ], + "interests" : { } + } + }, + "target" : { + "itemId" : "testFormTracking", + "itemType" : "form", + "scope" : "realEstateManager", + "version" : null, + "properties" : { } + }, + "persistent" : true +}</code></pre> +</div> +</div> +<div class="paragraph"> +<p>You can see in this event that the form values are sent as properties of the event itself, while the form name is sent +as the <code>target.itemId</code></p> +</div> +<div class="paragraph"> +<p>While setting up form tracking, it can be very useful to use the Apache Unomi Karaf SSH shell commands : <code>event-tail</code> +and <code>event-view</code> to check if you are properly receiving the form submission events and that they contain the expected +data. If not, check your tracking code for any errors.</p> +</div> +<div class="paragraph"> +<p>Now that the data is properly sent using an event to Apache Unomi, we must still use it to perform some kind of actions. +Using rules, we could do anything from updating the profile to sending the data to a third-party server (using a custom- +developped action of course). In this example we will illustrate how to update the profile.</p> +</div> +<div class="paragraph"> +<p>In order to do so we will deploy a rule that will copy data coming from the event into a profile. But we will need to +map the form field names to profile names, and this can be done using the <code>setPropertyAction</code> that’s available out of the +box in the Apache Unomi server.</p> +</div> +<div class="paragraph"> +<p>There are two ways to register rules : either by building a custom OSGi bundle plugin or using the REST API to directly +send a JSON representation of the rule to be saved. We will in this example use the CURL shell command to make a call to +the REST API.</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>curl -X POST -k -u karaf:karaf https://localhost:9443/cxs/rules \ + --header "Content-Type: application/json" \ +-d @- << EOF +{ + "itemId": "form-mapping-example", + "itemType": "rule", + "linkedItems": null, + "raiseEventOnlyOnceForProfile": false, + "raiseEventOnlyOnceForSession": false, + "priority": -1, + "metadata": { + "id": "form-mapping-example", + "name": "Example Form Mapping", + "description": "An example of how to map event properties to profile properties", + "scope": "realEstateManager", + "tags": [], + "enabled": true, + "missingPlugins": false, + "hidden": false, + "readOnly": false + }, + "condition": { + "type": "formEventCondition", + "parameterValues": { + "formId": "testFormTracking", + "pagePath" : "/tracker/" + } + }, + "actions": [ + { + "type": "setPropertyAction", + "parameterValues": { + "setPropertyName": "properties(firstName)", + "setPropertyValue": "eventProperty::properties(firstName)", + "setPropertyStrategy": "alwaysSet" + } + }, + { + "type": "setPropertyAction", + "parameterValues": { + "setPropertyName": "properties(lastName)", + "setPropertyValue": "eventProperty::properties(lastName)", + "setPropertyStrategy": "alwaysSet" + } + }, + { + "type": "setPropertyAction", + "parameterValues": { + "setPropertyName": "properties(email)", + "setPropertyValue": "eventProperty::properties(email)", + "setPropertyStrategy": "alwaysSet" + } + } + ] +} +EOF</code></pre> +</div> +</div> +<div class="paragraph"> +<p>As you can see in this request, we have a few parameters that need explaining:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p><code>-k</code> is used to accept any certificate as we are in this example using a default Apache Unomi server configuration that +comes with its predefined HTTPS certificates</p> +</li> +<li> +<p><code>-u karaf:karaf</code> is the default username/password for authenticating to the REST API. To change this value you should +edit the `etc/users.properties`file and it is required to modify this login before going to production.</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>Finally the rule itself should be pretty self-explanatory but there are a few important things to note :</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>the <code>itemId</code> and <code>metadata.id</code> values should be the same</p> +</li> +<li> +<p>the <code>scope</code> should be the same as the scope that was setup in the tracker initialization</p> +</li> +<li> +<p>the <code>formId</code> parameter must have the form name value</p> +</li> +<li> +<p>the <code>pagePath</code> should be the pagePath passed through the event (if you’re not sure of its value, you could either using +network debugging in the browser or use the <code>event-tail</code> and <code>event-view</code> commands in the Apache Unomi Karaf SSH shell).</p> +</li> +<li> +<p>the setPropertyAction may be repeated as many times as desired to copy the values from the event to the profile. Note that +the <code>setPropertyName</code> will define the property to set on the profile and the <code>setPropertyValue</code> will define where the +value is coming from. In this example the name and the value are the same but that is no way a requirement. It could +even be possible to using multiple <code>setPropertyAction</code> instances to copy the same event property into different profile +properties.</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>To check if your rule is properly deployed you can use the following SSH shell command :</p> +</div> +<div class="paragraph"> +<p><code>unomi:rule-view form-mapping-example</code></p> +</div> +<div class="paragraph"> +<p>The parameter is the <code>itemId</code> of the rule. If you want to see all the rules deployed in the system you can use the +command :</p> +</div> +<div class="paragraph"> +<p><code>unomi:rule-list 1000</code></p> +</div> +<div class="paragraph"> +<p>The <code>1000</code> parameter is the limit of number of objects to retrieve. As the number of rules can grow quickly in an Apache +Unomi instance, it is recommended to put this value a bit high to make sure you get the full list of rules.</p> +</div> +<div class="paragraph"> +<p>Once the rule is in place, try submitting the form with some values and check that the profile is properly updated. One +recommend way of doing this is to use the <code>event-tail</code> command that will output something like this :</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>ID |Type |Session |Profile |Timestamp |Scope |Persi| +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +cef09b89-6b99-4e4f-a99c-a4159a66b42b|form |aaad09aa-88c2-67bd-b106-5a47ded43ead|48563fd0-6319-4260-8dba-ae421beba26f|Fri Nov 23 17:52:33 CET 2018 |realEstateManag|true |</code></pre> +</div> +</div> +<div class="paragraph"> +<p>You can directly see the profile that is being used, so you can then simply use the</p> +</div> +<div class="paragraph"> +<p><code>unomi:profile-view 48563fd0-6319-4260-8dba-ae421beba26f</code></p> +</div> +<div class="paragraph"> +<p>command to see a JSON dump of the profile and check that the form values have been properly positioned.</p> +</div> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="_configuration">3. Configuration</h2> +<div class="sectionbody"> +<div class="sect2"> +<h3 id="_centralized_configuration">3.1. Centralized configuration</h3> +<div class="paragraph"> +<p>Apache Unomi uses a centralized configuration file that contains both system properties and configuration properties. +These settings are then fed to the OSGi and other configuration files using placeholder that look something like this:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>contextserver.publicAddress=${org.apache.unomi.cluster.public.address:-http://localhost:8181} +contextserver.internalAddress=${org.apache.unomi.cluster.internal.address:-https://localhost:9443}</code></pre> +</div> +</div> +<div class="paragraph"> +<p>Default values are stored in a file called <code>$MY_KARAF_HOME/etc/custom.system.properties</code> but you should never modify +this file directly, as an override mechanism is available. Simply create a file called:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>unomi.custom.system.properties</pre> +</div> +</div> +<div class="paragraph"> +<p>and put your own property values in their to override the defaults OR you can use environment variables to also override +the values in the <code>$MY_KARAF_HOME/etc/custom.system.properties</code>. See the next section for more information about that.</p> +</div> +</div> +<div class="sect2"> +<h3 id="_changing_the_default_configuration_using_environment_variables_i_e_docker_configuration">3.2. Changing the default configuration using environment variables (i.e. Docker configuration)</h3> +<div class="paragraph"> +<p>You might want to use environment variables to change the default system configuration, especially if you intend to run +Apache Unomi inside a Docker container. You can find the list of all the environment variable names in the following file:</p> +</div> +<div class="paragraph"> +<p><a href="https://github.com/apache/unomi/blob/master/package/src/main/resources/etc/custom.system.properties"; class="bare">https://github.com/apache/unomi/blob/master/package/src/main/resources/etc/custom.system.properties</a></p> +</div> +<div class="paragraph"> +<p>If you are using Docker Container, simply pass the environment variables on the docker command line or if you are using +Docker Compose you can put the environment variables in the docker-compose.yml file.</p> +</div> +<div class="paragraph"> +<p>If you want to "save" the environment values in a file, you can use the <code>bin/setenv(.bat)</code> to setup the environment +variables you want to use.</p> +</div> +</div> +<div class="sect2"> +<h3 id="_changing_the_default_configuration_using_property_files">3.3. Changing the default configuration using property files</h3> +<div class="paragraph"> +<p>If you want to change the default configuration using property files instead of environment variables, you can perform +any modification you want in the <code>$MY_KARAF_HOME/etc/unomi.custom.system.properties</code> file.</p> +</div> +<div class="paragraph"> +<p>By default this file does not exist and is designed to be a file that will contain only your custom modifications to the +default configuration.</p> +</div> +<div class="paragraph"> +<p>For example, if you want to change the HTTP ports that the server is listening on, you will need to create the +following lines in the $MY_KARAF_HOME/etc/unomi.custom.system.properties (and create it if you haven’t yet) file:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>org.osgi.service.http.port.secure=9443 +org.osgi.service.http.port=8181</code></pre> +</div> +</div> +<div class="paragraph"> +<p>If you change these ports, also make sure you adjust the following settings in the same file :</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>org.apache.unomi.cluster.public.address=http://localhost:8181 +org.apache.unomi.cluster.internal.address=https://localhost:9443</code></pre> +</div> +</div> +<div class="paragraph"> +<p>If you need to specify an ElasticSearch cluster name, or a host and port that are different than the default, +it is recommended to do this BEFORE you start the server for the first time, or you will loose all the data +you have stored previously.</p> +</div> +<div class="paragraph"> +<p>You can use the following properties for the ElasticSearch configuration</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>org.apache.unomi.elasticsearch.cluster.name=contextElasticSearch +# The elasticsearch.adresses may be a comma seperated list of host names and ports such as +# hostA:9200,hostB:9200 +# Note: the port number must be repeated for each host. +org.apache.unomi.elasticsearch.addresses=localhost:9200</code></pre> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="_secured_events_configuration">3.4. Secured events configuration</h3> +<div class="paragraph"> +<p>Apache Unomi secures some events by default. It comes out of the box with a default configuration that you can adjust +by using the centralized configuration file override in <code>$MY_KARAF_HOME/etc/unomi.custom.system.properties</code></p> +</div> +<div class="paragraph"> +<p>You can find the default configuration in the following file:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>$MY_KARAF_HOME/etc/custom.system.properties</pre> +</div> +</div> +<div class="paragraph"> +<p>The properties start with the prefix : <code>org.apache.unomi.thirdparty.*</code> and here are the default values :</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>org.apache.unomi.thirdparty.provider1.key=${env:UNOMI_THIRDPARTY_PROVIDER1_KEY:-670c26d1cc413346c3b2fd9ce65dab41} +org.apache.unomi.thirdparty.provider1.ipAddresses=${env:UNOMI_THIRDPARTY_PROVIDER1_IPADDRESSES:-127.0.0.1,::1} +org.apache.unomi.thirdparty.provider1.allowedEvents=${env:UNOMI_THIRDPARTY_PROVIDER1_ALLOWEDEVENTS:-login,updateProperties}</pre> +</div> +</div> +<div class="paragraph"> +<p>The events set in allowedEvents will be secured and will only be accepted if the call comes from the specified IP +address, and if the secret-key is passed in the X-Unomi-Peer HTTP request header. The "env:" part means that it will +attempt to read an environment variable by that name, and if it’s not found it will default to the value after the ":-" +marker.</p> +</div> +<div class="paragraph"> +<p>It is now also possible to use IP address ranges instead of having to list all valid IP addresses for event sources. This +is very useful when working in cluster deployments where servers may be added or removed dynamically. In order to support +this Apache Unomi uses a library called <a href="https://seancfoley.github.io/IPAddress/#_Toc525135541";>IPAddress</a> that supports +IP ranges and subnets. Here is an example of how to setup a range:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>org.apache.unomi.thirdparty.provider1.ipAddresses=${env:UNOMI_THIRDPARTY_PROVIDER1_IPADDRESSES:-192.168.1.1-100,::1}</pre> +</div> +</div> +<div class="paragraph"> +<p>The above configuration will allow a range of IP addresses between 192.168.1.1 and 192.168.1.100 as well as the IPv6 +loopback.</p> +</div> +<div class="paragraph"> +<p>Here’s another example using the subnet format:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>org.apache.unomi.thirdparty.provider1.ipAddresses=${env:UNOMI_THIRDPARTY_PROVIDER1_IPADDRESSES:-1.2.0.0/16,::1}</pre> +</div> +</div> +<div class="paragraph"> +<p>The above configuration will allow all addresses starting with 1.2 as well as the IPv6 loopback address.</p> +</div> +<div class="paragraph"> +<p>Wildcards may also be used:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>org.apache.unomi.thirdparty.provider1.ipAddresses=${env:UNOMI_THIRDPARTY_PROVIDER1_IPADDRESSES:-1.2.*.*,::1}</pre> +</div> +</div> +<div class="paragraph"> +<p>The above configuration is exactly the same as the previous one.</p> +</div> +<div class="paragraph"> +<p>More advanced ranges and subnets can be used as well, please refer to the <a href="https://seancfoley.github.io/IPAddress";>IPAddress</a> library documentation for details on +how to format them.</p> +</div> +<div class="paragraph"> +<p>If you want to add another provider you will need to add them manually in the following file (and make sure you maintain +the changes when upgrading) :</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>$MY_KARAF_HOME/etc/org.apache.unomi.thirdparty.cfg</pre> +</div> +</div> +<div class="paragraph"> +<p>Usually, login events, which operate on profiles and do merge on protected properties, must be secured. For each +trusted third party server, you need to add these 3 lines :</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>thirdparty.provider1.key=secret-key +thirdparty.provider1.ipAddresses=127.0.0.1,::1 +thirdparty.provider1.allowedEvents=login,updateProperties</code></pre> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="_installing_the_maxmind_geoiplite2_ip_lookup_database">3.5. Installing the MaxMind GeoIPLite2 IP lookup database</h3> +<div class="paragraph"> +<p>Apache Unomi requires an IP database in order to resolve IP addresses to user location. +The GeoLite2 database can be downloaded from MaxMind here : +<a href="http://dev.maxmind.com/geoip/geoip2/geolite2/";>http://dev.maxmind.com/geoip/geoip2/geolite2/</a></p> +</div> +<div class="paragraph"> +<p>Simply download the GeoLite2-City.mmdb file into the "etc" directory.</p> +</div> +</div> +<div class="sect2"> +<h3 id="_installing_geonames_database">3.6. Installing Geonames database</h3> +<div class="paragraph"> +<p>Apache Unomi includes a geocoding service based on the geonames database ( <a href="http://www.geonames.org/";>http://www.geonames.org/</a> ). It can be +used to create conditions on countries or cities.</p> +</div> +<div class="paragraph"> +<p>In order to use it, you need to install the Geonames database into . Get the "allCountries.zip" database from here : +<a href="http://download.geonames.org/export/dump/";>http://download.geonames.org/export/dump/</a></p> +</div> +<div class="paragraph"> +<p>Download it and put it in the "etc" directory, without unzipping it. +Edit <code>$MY_KARAF_HOME/etc/unomi.custom.system.properties</code> and set <code>org.apache.unomi.geonames.forceImport</code> to true, +import should start right away. +Otherwise, import should start at the next startup. Import runs in background, but can take about 15 minutes. +At the end, you should have about 4 million entries in the geonames index.</p> +</div> +</div> +<div class="sect2"> +<h3 id="_rest_api_security">3.7. REST API Security</h3> +<div class="paragraph"> +<p>The Apache Unomi Context Server REST API is protected using JAAS authentication and using Basic or Digest HTTP auth. +By default, the login/password for the REST API full administrative access is "karaf/karaf".</p> +</div> +<div class="paragraph"> +<p>The generated package is also configured with a default SSL certificate. You can change it by following these steps :</p> +</div> +<div class="paragraph"> +<p>Replace the existing keystore in $MY_KARAF_HOME/etc/keystore by your own certificate :</p> +</div> +<div class="paragraph"> +<p><a href="http://wiki.eclipse.org/Jetty/Howto/Configure_SSL";>http://wiki.eclipse.org/Jetty/Howto/Configure_SSL</a></p> +</div> +<div class="paragraph"> +<p>Update the keystore and certificate password in $MY_KARAF_HOME/etc/unomi.custom.system.properties file :</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>org.ops4j.pax.web.ssl.keystore=${env:UNOMI_SSL_KEYSTORE:-${karaf.etc}/keystore} +org.ops4j.pax.web.ssl.password=${env:UNOMI_SSL_PASSWORD:-changeme} +org.ops4j.pax.web.ssl.keypassword=${env:UNOMI_SSL_KEYPASSWORD:-changeme}</code></pre> +</div> +</div> +<div class="paragraph"> +<p>You should now have SSL setup on Karaf with your certificate, and you can test it by trying to access it on port 9443.</p> +</div> +<div class="paragraph"> +<p>Changing the default Karaf password can be done by modifying the <code>org.apache.unomi.security.root.password</code> in the +<code>$MY_KARAF_HOME/etc/unomi.custom.system.properties</code> file</p> +</div> +</div> +<div class="sect2"> +<h3 id="_scripting_security">3.8. Scripting security</h3> +<div class="sect3"> +<h4 id="_multi_layer_scripting_filtering_system">3.8.1. Multi-layer scripting filtering system</h4> +<div class="paragraph"> +<p>The scripting security system is multi-layered.</p> +</div> +<div class="paragraph"> +<p>For requests coming in through the /cxs/context.json endpoint, the following flow is used to secure incoming requests:</p> +</div> +<div class="imageblock"> +<div class="content"> +<img src="images/expression-filtering-layers.png" alt="Expression filtering layers"> +</div> +</div> +<div class="paragraph"> +<p>Conditions submitted through the context.json public endpoint are first sanitized, meaning that any scripting directly +injected is removed. However, as conditions can use sub conditions that include scripting, only the first directly +injected layer of scripts are removed.</p> +</div> +<div class="paragraph"> +<p>The second layer is the expression filtering system, that uses an allow-listing mechanism to only accept pre-vetted +expressions (through configuration and deployment on the server side). Any unrecognized expression will not be accepted.</p> +</div> +<div class="paragraph"> +<p>Finally, once the script starts executing in the scripting engine, a filtering class loader will only let the script +access classes that have been allowed.</p> +</div> +<div class="paragraph"> +<p>This multi-layered approach makes it possible to retain a high level of security even if one layer is poorly +configured or abused.</p> +</div> +<div class="paragraph"> +<p>For requests coming in through the secure APIs such as rules, only the condition sanitizing step is skipped, +otherwise the rest of the filtering system is the same.</p> +</div> +</div> +<div class="sect3"> +<h4 id="_scripts_and_expressions">3.8.2. Scripts and expressions</h4> +<div class="paragraph"> +<p>Apache Unomi allows using different types of expressions in the following subsystems:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>context.json filters and personalization queries</p> +</li> +<li> +<p>rule conditions and actions parameters</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>Apache Unomi uses two integrated scripting languages to provide this functionality: OGNL and MVEL. +OGNL is deprecated and is now disabled by default since 1.5.2 as it is little used (and replaced by better performing +hardcoded property lookups). MVEL is more commonly used in rule actions as in the following example:</p> +</div> +<div class="paragraph"> +<p>From <a href="https://github.com/apache/unomi/blob/unomi-1.5.x/plugins/baseplugin/src/main/resources/META-INF/cxs/rules/sessionAssigned.json";>https://github.com/apache/unomi/blob/unomi-1.5.x/plugins/baseplugin/src/main/resources/META-INF/cxs/rules/sessionAssigned.json</a>:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code class="language-json" data-lang="json">{ + "metadata": { + "id": "_ajhg9u2s5_sessionAssigned", + "name": "Session assigned to a profile", + "description": "Update profile visit information", + "readOnly":true + }, + + "condition": { + "type": "booleanCondition", + "parameterValues": { + "subConditions":[ + { + "type": "eventTypeCondition", + "parameterValues": { + "eventTypeId": "sessionCreated" + } + }, + { + "type": "eventTypeCondition", + "parameterValues": { + "eventTypeId": "sessionReassigned" + } + } + + ], + "operator":"or" + + } + }, + + "actions": [ + { + "parameterValues": { + "setPropertyName": "properties.previousVisit", + "setPropertyValue": "profileProperty::lastVisit", + "storeInSession": false + }, + "type": "setPropertyAction" + }, + { + "parameterValues": { + "setPropertyName": "properties.lastVisit", + "setPropertyValue": "now", + "storeInSession": false + }, + "type": "setPropertyAction" + }, + { + "parameterValues": { + "setPropertyName": "properties.nbOfVisits", + "setPropertyValue": "script::profile.properties.?nbOfVisits != null ? (profile.properties.nbOfVisits + 1) : 1", + "storeInSession": false + }, + "type": "setPropertyAction" + } + ] + +}</code></pre> +</div> +</div> +<div class="paragraph"> +<p>As we see in the above example, we use an MVEL script with the setPropertyAction to set a property value. +Starting with version 1.5.2, any expression use in rules MUST be allow-listed.</p> +</div> +<div class="paragraph"> +<p>OGNL was previously used wherever a parameter could be used, but MVEL could only be used with a âscript::â prefix. +Starting with version 1.5.2 OGNL will no longer be allowed and is replaced by a compatible âhardcodedâ property +lookup system, while MVEL requires allow-listing the scripts that are to be used.</p> +</div> +<div class="paragraph"> +<p>By default, Apache Unomi comes with some built-in allowed expressions that cover all the internal uses cases.</p> +</div> +<div class="paragraph"> +<p>Default allowed MVEL expressions (from <a href="https://github.com/apache/unomi/blob/unomi-1.5.x/plugins/baseplugin/src/main/resources/META-INF/cxs/expressions/mvel.json";>https://github.com/apache/unomi/blob/unomi-1.5.x/plugins/baseplugin/src/main/resources/META-INF/cxs/expressions/mvel.json</a>) :</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code class="language-json" data-lang="json">[ + "\\Q'systemProperties.goals.'+goalId+'TargetReached'\\E", + "\\Q'now-'+since+'d'\\E", + "\\Q'scores.'+scoringPlanId\\E", + "\\QminimumDuration*1000\\E", + "\\QmaximumDuration*1000\\E", + "\\Qprofile.properties.?nbOfVisits != null ? (profile.properties.nbOfVisits + 1) : 1\\E", + "\\Qsession != null ? session.size + 1 : 0\\E", + "\\Q'properties.optimizationTest_'+event.target.itemId\\E", + "\\Qevent.target.properties.variantId\\E", + "\\Qprofile.properties.?systemProperties.goals.\\E[\\w\\_]*\\QReached != null ? (profile.properties.systemProperties.goals.\\E[\\w\\_]*\\QReached) : 'now'\\E", + "\\Qprofile.properties.?systemProperties.campaigns.\\E[\\w\\_]*\\QEngaged != null ? (profile.properties.systemProperties.campaigns.\\E[\\w\\_]*\\QEngaged) : 'now'\\E" +]</code></pre> +</div> +</div> +<div class="paragraph"> +<p>If you require or are already using custom expressions, you should add a plugin to Apache Unomi to allow for this.
[... 7417 lines stripped ...]
