http://git-wip-us.apache.org/repos/asf/atlas-website/blob/c3e5130b/Configuration.html ---------------------------------------------------------------------- diff --git a/Configuration.html b/Configuration.html index ff75f66..bd09e28 100644 --- a/Configuration.html +++ b/Configuration.html @@ -1,325 +1,154 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia at 2017-08-09 - | Rendered using Apache Maven Fluido Skin 1.3.0 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/twiki/Configuration.twiki at 2018-01-25 + | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20170809" /> + <meta name="Date-Revision-yyyymmdd" content="20180125" /> <meta http-equiv="Content-Language" content="en" /> <title>Apache Atlas – Configuring Apache Atlas - Application Properties</title> - <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.7.min.css" /> <link rel="stylesheet" href="./css/site.css" /> <link rel="stylesheet" href="./css/print.css" media="print" /> - - - <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script> - - - -<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> - - </head> - <body class="topBarEnabled"> - - - - - + <script type="text/javascript" src="./js/apache-maven-fluido-1.7.min.js"></script> + </head> + <body class="topBarEnabled"> <div id="topbar" class="navbar navbar-fixed-top "> <div class="navbar-inner"> - <div class="container" style="width: 68%;"><div class="nav-collapse"> - - - <ul class="nav"> - <li class="dropdown"> + <div class="container" style="width: 68%;"><div class="nav-collapse"> + <ul class="nav"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Atlas <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="index.html" title="About">About</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/ATLAS"; title="Wiki">Wiki</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/ATLAS"; title="News">News</a> -</li> - - <li> <a href="https://git-wip-us.apache.org/repos/asf/atlas.git"; title="Git">Git</a> -</li> - - <li> <a href="https://issues.apache.org/jira/browse/ATLAS"; title="Jira">Jira</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/ATLAS/PoweredBy"; title="Powered by">Powered by</a> -</li> - - <li> <a href="http://blogs.apache.org/atlas/"; title="Blog">Blog</a> -</li> - </ul> + <li><a href="index.html" title="About">About</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/ATLAS"; title="Wiki">Wiki</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/ATLAS"; title="News">News</a></li> + <li><a href="https://git-wip-us.apache.org/repos/asf/atlas.git"; title="Git">Git</a></li> + <li><a href="https://issues.apache.org/jira/browse/ATLAS"; title="Jira">Jira</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/ATLAS/PoweredBy"; title="Powered by">Powered by</a></li> + <li><a href="http://blogs.apache.org/atlas/"; title="Blog">Blog</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Project Information <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="project-info.html" title="Summary">Summary</a> -</li> - - <li> <a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a> -</li> - - <li> <a href="http://webchat.freenode.net?channels=apacheatlas&uio=d4"; title="IRC">IRC</a> -</li> - - <li> <a href="team-list.html" title="Team">Team</a> -</li> - - <li> <a href="issue-tracking.html" title="Issue Tracking">Issue Tracking</a> -</li> - - <li> <a href="source-repository.html" title="Source Repository">Source Repository</a> -</li> - - <li> <a href="license.html" title="License">License</a> -</li> - </ul> + <li><a href="project-info.html" title="Summary">Summary</a></li> + <li><a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a></li> + <li><a href="http://webchat.freenode.net?channels=apacheatlas&uio=d4"; title="IRC">IRC</a></li> + <li><a href="team-list.html" title="Team">Team</a></li> + <li><a href="issue-tracking.html" title="Issue Tracking">Issue Tracking</a></li> + <li><a href="source-repository.html" title="Source Repository">Source Repository</a></li> + <li><a href="license.html" title="License">License</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Releases <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="http://www.apache.org/dyn/closer.cgi/atlas/0.8.0-incubating/"; title="0.8-incubating">0.8-incubating</a> -</li> - - <li> <a href="http://archive.apache.org/dist/incubator/atlas/0.7.1-incubating/"; title="0.7.1-incubating">0.7.1-incubating</a> -</li> - - <li> <a href="http://archive.apache.org/dist/incubator/atlas/0.7.0-incubating/"; title="0.7-incubating">0.7-incubating</a> -</li> - - <li> <a href="http://archive.apache.org/dist/incubator/atlas/0.6.0-incubating/"; title="0.6-incubating">0.6-incubating</a> -</li> - - <li> <a href="http://archive.apache.org/dist/incubator/atlas/0.5.0-incubating/"; title="0.5-incubating">0.5-incubating</a> -</li> - </ul> + <li><a href="http://www.apache.org/dyn/closer.cgi/atlas/1.0.0-alpha/"; title="1.0.0-alpha">1.0.0-alpha</a></li> + <li><a href="http://www.apache.org/dyn/closer.cgi/atlas/0.8.1/"; title="0.8.1">0.8.1</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.8.0-incubating/"; title="0.8-incubating">0.8-incubating</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.7.1-incubating/"; title="0.7.1-incubating">0.7.1-incubating</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.7.0-incubating/"; title="0.7-incubating">0.7-incubating</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.6.0-incubating/"; title="0.6-incubating">0.6-incubating</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.5.0-incubating/"; title="0.5-incubating">0.5-incubating</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentation <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="0.8.0-incubating/index.html" title="0.8-incubating">0.8-incubating</a> -</li> - - <li> <a href="0.7.1-incubating/index.html" title="0.7.1-incubating">0.7.1-incubating</a> -</li> - - <li> <a href="0.7.0-incubating/index.html" title="0.7-incubating">0.7-incubating</a> -</li> - - <li> <a href="0.6.0-incubating/index.html" title="0.6-incubating">0.6-incubating</a> -</li> - - <li> <a href="0.5.0-incubating/index.html" title="0.5-incubating">0.5-incubating</a> -</li> - </ul> + <li><a href="1.0.0-alpha/index.html" title="1.0.0-alpha">1.0.0-alpha</a></li> + <li><a href="0.8.1/index.html" title="0.8.1">0.8.1</a></li> + <li><a href="0.8.0-incubating/index.html" title="0.8-incubating">0.8-incubating</a></li> + <li><a href="0.7.1-incubating/index.html" title="0.7.1-incubating">0.7.1-incubating</a></li> + <li><a href="0.7.0-incubating/index.html" title="0.7-incubating">0.7-incubating</a></li> + <li><a href="0.6.0-incubating/index.html" title="0.6-incubating">0.6-incubating</a></li> + <li><a href="0.5.0-incubating/index.html" title="0.5-incubating">0.5-incubating</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">ASF <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="http://www.apache.org/foundation/how-it-works.html"; title="How Apache Works">How Apache Works</a> -</li> - - <li> <a href="http://www.apache.org/foundation/"; title="Foundation">Foundation</a> -</li> - - <li> <a href="http://www.apache.org/foundation/sponsorship.html"; title="Sponsoring Apache">Sponsoring Apache</a> -</li> - - <li> <a href="http://www.apache.org/foundation/thanks.html"; title="Thanks">Thanks</a> -</li> - </ul> + <li><a href="http://www.apache.org/foundation/how-it-works.html"; title="How Apache Works">How Apache Works</a></li> + <li><a href="http://www.apache.org/foundation/"; title="Foundation">Foundation</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html"; title="Sponsoring Apache">Sponsoring Apache</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html"; title="Thanks">Thanks</a></li> + </ul> </li> - </ul> - - <form id="search-form" action="http://www.google.com/search"; method="get" class="navbar-search pull-right" > - + </ul> +<form id="search-form" action="https://www.google.com/search"; method="get" class="navbar-search pull-right" > <input value="http://atlas.apache.org"; name="sitesearch" type="hidden"/> <input class="search-query" name="q" id="query" type="text" /> </form> -<script type="text/javascript" src="http://www.google.com/coop/cse/brand?form=search-form";></script> - - - - - - <iframe src="http://www.facebook.com/plugins/like.php?href=http://atlas.apache.org/atlas-docs&send=false&layout=button_count&show-faces=false&action=like&colorscheme=dark"; +<script type="text/javascript">asyncJs( 'https://cse.google.com/brand?form=search-form' )</script> + <iframe src="https://www.facebook.com/plugins/like.php?href=http://atlas.apache.org/atlas-docs&send=false&layout=button_count&show-faces=false&action=like&colorscheme=dark"; scrolling="no" frameborder="0" - style="border:none; width:80px; height:20px; margin-top: 10px;" class="pull-right" ></iframe> - - <script type="text/javascript" src="https://apis.google.com/js/plusone.js";></script> - - <ul class="nav pull-right"><li style="margin-top: 10px;"> - + style="border:none; width:100px; height:20px; margin-top: 10px;" class="pull-right" ></iframe> + <script type="text/javascript">asyncJs( 'https://apis.google.com/js/plusone.js' )</script> + <ul class="nav pull-right"><li style="margin-top: 10px;"> <div class="g-plusone" data-href="http://atlas.apache.org/atlas-docs"; data-size="medium" width="60px" align="right" ></div> - - </li></ul> - - - </div> - + </li></ul> + </div> </div> </div> </div> - - <div class="container"> - <div id="banner"> - <div class="pull-left"> - <a href=".." id="bannerLeft"> - <img src="images/atlas-logo.png" alt="Apache Atlas" width="200px" height="45px"/> - </a> - </div> - <div class="pull-right"> </div> + <div class="container"> + <div id="banner"> + <div class="pull-left"><a href=".." id="bannerLeft"><img src="images/atlas-logo.png" alt="Apache Atlas" width="200px" height="45px"/></a></div> + <div class="pull-right"></div> <div class="clear"><hr/></div> </div> <div id="breadcrumbs"> <ul class="breadcrumb"> - - - <li class=""> - <a href="http://www.apache.org"; class="externalLink" title="Apache"> - Apache</a> - </li> - <li class="divider ">/</li> - <li class=""> - <a href="index.html" title="Atlas"> - Atlas</a> - </li> - <li class="divider ">/</li> - <li class="">Configuring Apache Atlas - Application Properties</li> - - - - <li id="publishDate" class="pull-right">Last Published: 2017-08-09</li> <li class="divider pull-right">|</li> - <li id="projectVersion" class="pull-right">Version: 0.9-SNAPSHOT</li> - - </ul> + <li class=""><a href="http://www.apache.org"; class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li> + <li class=""><a href="index.html" title="Atlas">Atlas</a><span class="divider">/</span></li> + <li class="active ">Configuring Apache Atlas - Application Properties</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-01-25</li> + <li id="projectVersion" class="pull-right">Version: 1.0.0-alpha</li> + </ul> </div> - - - <div id="bodyColumn" > - - <div class="section"> +<div class="section"> <h2><a name="Configuring_Apache_Atlas_-_Application_Properties"></a>Configuring Apache Atlas - Application Properties</h2> <p>All configuration in Atlas uses java properties style configuration. The main configuration file is atlas-application.properties which is in the <b>conf</b> dir at the deployed location. It consists of the following sections:</p></div> <div class="section"> <h3><a name="Graph_Configs"></a>Graph Configs</h3></div> <div class="section"> -<h4><a name="Graph_persistence_engine"></a>Graph persistence engine</h4> -<p>This section sets up the graph db - titan - to use a persistence engine. Please refer to <a href="http://s3.thinkaurelius.com/docs/titan/0.5.4/titan-config-ref.html";>link</a> for more details. The example below uses BerkeleyDBJE.</p> -<div class="source"> -<pre> -atlas.graph.storage.backend=berkeleyje -atlas.graph.storage.directory=data/berkley - -</pre></div></div> -<div class="section"> -<h5><a name="Graph_persistence_engine_-_Hbase"></a>Graph persistence engine - Hbase</h5> -<p>Basic configuration</p> -<div class="source"> -<pre> +<h4><a name="Graph_Persistence_engine_-_HBase"></a>Graph Persistence engine - HBase</h4> +<p>Set the following properties to configure <a href="./JanusGraph.html">JanusGraph</a> to use HBase as the persistence engine. Please refer to <a href="http://docs.janusgraph.org/0.2.0/configuration.html#_hbase_caching";>link</a> for more details.</p> +<div class="source"><pre class="prettyprint"> atlas.graph.storage.backend=hbase -#For standalone mode , specify localhost -#for distributed mode, specify zookeeper quorum here - For more information refer http://s3.thinkaurelius.com/docs/titan/current/hbase.html#_remote_server_mode_2 atlas.graph.storage.hostname=<ZooKeeper Quorum> +atlas.graph.storage.hbase.table=atlas </pre></div> -<p>HBASE_CONF_DIR environment variable needs to be set to point to the Hbase client configuration directory which is added to classpath when Atlas starts up. hbase-site.xml needs to have the following properties set according to the cluster setup</p> -<div class="source"> -<pre> -#Set below to /hbase-secure if the Hbase server is setup in secure mode -zookeeper.znode.parent=/hbase-unsecure - -</pre></div> -<p>Advanced configuration</p> -<p># If you are planning to use any of the configs mentioned below, they need to be prefixed with "atlas.graph." to take effect in ATLAS Refer <a class="externalLink" href="http://s3.thinkaurelius.com/docs/titan/0.5.4/titan-config-ref.html#_storage_hbase";>http://s3.thinkaurelius.com/docs/titan/0.5.4/titan-config-ref.html#_storage_hbase</a></p> -<p>Permissions</p> -<p>When Atlas is configured with HBase as the storage backend the graph db (titan) needs sufficient user permissions to be able to create and access an HBase table. In a secure cluster it may be necessary to grant permissions to the 'atlas' user for the 'titan' table.</p> -<p>With Ranger, a policy can be configured for 'titan'.</p> -<p>Without Ranger, HBase shell can be used to set the permissions.</p> -<div class="source"> -<pre> - su hbase - kinit -k -t <hbase keytab> <hbase principal> - echo "grant 'atlas', 'RWXCA', 'titan'" | hbase shell - -</pre></div> -<p>Note that if the embedded-hbase-solr profile is used then HBase is included in the distribution so that a standalone instance of HBase can be started as the default storage backend for the graph repository. Using the embedded-hbase-solr profile will configure Atlas so that HBase instance will be started and stopped along with the Atlas server by default. To use the embedded-hbase-solr profile please see "Building Atlas" in the <a href="./InstallationSteps.html">Installation Steps</a> section.</p></div> +<p>If any further <a href="./JanusGraph.html">JanusGraph</a> configuration needs to be setup, please prefix the property name with "atlas.graph.".</p> +<p>In addition to setting up configurations, please ensure that environment variable HBASE_CONF_DIR is setup to point to the directory containing HBase configuration file hbase-site.xml.</p></div> <div class="section"> -<h4><a name="Graph_Search_Index"></a>Graph Search Index</h4> -<p>This section sets up the graph db - titan - to use an search indexing system. The example configuration below sets up to use an embedded Elastic search indexing system.</p> -<div class="source"> -<pre> -atlas.graph.index.search.backend=elasticsearch -atlas.graph.index.search.directory=data/es -atlas.graph.index.search.elasticsearch.client-only=false -atlas.graph.index.search.elasticsearch.local-mode=true -atlas.graph.index.search.elasticsearch.create.sleep=2000 +<h4><a name="Graph_Search_Index_-_Solr"></a>Graph Search Index - Solr</h4> +<p>Solr installation in Cloud mode is a prerequisite for Apache Atlas use. Set the following properties to configure <a href="./JanusGraph.html">JanusGraph</a> to use Solr as the index search engine.</p> +<div class="source"><pre class="prettyprint"> +atlas.graph.index.search.backend=solr5 +atlas.graph.index.search.solr.mode=cloud +atlas.graph.index.search.solr.wait-searcher=true -</pre></div></div> -<div class="section"> -<h5><a name="Graph_Search_Index_-_Solr"></a>Graph Search Index - Solr</h5> -<p>Please note that Solr installation in Cloud mode is a prerequisite before configuring Solr as the search indexing backend. Refer <a href="./InstallationSteps.html">InstallationSteps</a> section for Solr installation/configuration.</p> -<div class="source"> -<pre> - atlas.graph.index.search.backend=solr5 - atlas.graph.index.search.solr.mode=cloud - atlas.graph.index.search.solr.zookeeper-url=<the ZK quorum setup for solr as comma separated value> eg: 10.1.6.4:2181,10.1.6.5:2181 - atlas.graph.index.search.solr.zookeeper-connect-timeout=<SolrCloud Zookeeper Connection Timeout>. Default value is 60000 ms - atlas.graph.index.search.solr.zookeeper-session-timeout=<SolrCloud Zookeeper Session Timeout>. Default value is 60000 ms +# ZK quorum setup for solr as comma separated value. Example: 10.1.6.4:2181,10.1.6.5:2181 +atlas.graph.index.search.solr.zookeeper-url= -</pre></div> -<p>Also note that if the embedded-hbase-solr profile is used then Solr is included in the distribution so that a standalone instance of Solr can be started as the default search indexing backend. Using the embedded-hbase-solr profile will configure Atlas so that the standalone Solr instance will be started and stopped along with the Atlas server by default. To use the embedded-hbase-solr profile please see "Building Atlas" in the <a href="./InstallationSteps.html">Installation Steps</a> section.</p></div> -<div class="section"> -<h4><a name="Choosing_between_Persistence_and_Indexing_Backends"></a>Choosing between Persistence and Indexing Backends</h4> -<p>Refer <a class="externalLink" href="http://s3.thinkaurelius.com/docs/titan/0.5.4/bdb.html";>http://s3.thinkaurelius.com/docs/titan/0.5.4/bdb.html</a> and <a class="externalLink" href="http://s3.thinkaurelius.com/docs/titan/0.5.4/hbase.html";>http://s3.thinkaurelius.com/docs/titan/0.5.4/hbase.html</a> for choosing between the persistence backends. BerkeleyDB is suitable for smaller data sets in the range of upto 10 million vertices with ACID gurantees. HBase on the other hand doesnt provide ACID guarantees but is able to scale for larger graphs. HBase also provides HA inherently.</p></div> -<div class="section"> -<h4><a name="Choosing_between_Persistence_Backends"></a>Choosing between Persistence Backends</h4> -<p>Refer <a class="externalLink" href="http://s3.thinkaurelius.com/docs/titan/0.5.4/bdb.html";>http://s3.thinkaurelius.com/docs/titan/0.5.4/bdb.html</a> and <a class="externalLink" href="http://s3.thinkaurelius.com/docs/titan/0.5.4/hbase.html";>http://s3.thinkaurelius.com/docs/titan/0.5.4/hbase.html</a> for choosing between the persistence backends. BerkeleyDB is suitable for smaller data sets in the range of upto 10 million vertices with ACID gurantees. HBase on the other hand doesnt provide ACID guarantees but is able to scale for larger graphs. HBase also provides HA inherently.</p></div> -<div class="section"> -<h4><a name="Choosing_between_Indexing_Backends"></a>Choosing between Indexing Backends</h4> -<p>Refer <a class="externalLink" href="http://s3.thinkaurelius.com/docs/titan/0.5.4/elasticsearch.html";>http://s3.thinkaurelius.com/docs/titan/0.5.4/elasticsearch.html</a> and <a class="externalLink" href="http://s3.thinkaurelius.com/docs/titan/0.5.4/solr.html";>http://s3.thinkaurelius.com/docs/titan/0.5.4/solr.html</a> for choosing between ElasticSearch and Solr. Solr in cloud mode is the recommended setup.</p></div> -<div class="section"> -<h4><a name="Switching_Persistence_Backend"></a>Switching Persistence Backend</h4> -<p>For switching the storage backend from BerkeleyDB to HBase and vice versa, refer the documentation for "Graph Persistence Engine" described above and restart ATLAS. The data in the indexing backend needs to be cleared else there will be discrepancies between the storage and indexing backend which could result in errors during the search. ElasticSearch runs by default in embedded mode and the data could easily be cleared by deleting the ATLAS_HOME/data/es directory. For Solr, the collections which were created during ATLAS Installation - vertex_index, edge_index, fulltext_index could be deleted which will cleanup the indexes</p></div> -<div class="section"> -<h4><a name="Switching_Index_Backend"></a>Switching Index Backend</h4> -<p>Switching the Index backend requires clearing the persistence backend data. Otherwise there will be discrepancies between the persistence and index backends since switching the indexing backend means index data will be lost. This leads to "Fulltext" queries not working on the existing data For clearing the data for BerkeleyDB, delete the ATLAS_HOME/data/berkeley directory For clearing the data for HBase, in Hbase shell, run 'disable titan' and 'drop titan'</p></div> -<div class="section"> -<h3><a name="Lineage_Configs"></a>Lineage Configs</h3> -<p>The higher layer services like lineage, schema, etc. are driven by the type system and this section encodes the specific types for the hive data model.</p> -<p># This models reflects the base super types for Data and Process</p> -<div class="source"> -<pre> -atlas.lineage.hive.table.type.name=DataSet -atlas.lineage.hive.process.type.name=Process -atlas.lineage.hive.process.inputs.name=inputs -atlas.lineage.hive.process.outputs.name=outputs - -## Schema -atlas.lineage.hive.table.schema.query=hive_table where name=?, columns +# SolrCloud Zookeeper Connection Timeout. Default value is 60000 ms +atlas.graph.index.search.solr.zookeeper-connect-timeout=60000 + +# SolrCloud Zookeeper Session Timeout. Default value is 60000 ms +atlas.graph.index.search.solr.zookeeper-session-timeout=60000 </pre></div></div> <div class="section"> <h3><a name="Search_Configs"></a>Search Configs</h3> -<p>Search APIs (DSL and full text search) support pagination and have optional limit and offset arguments. Following configs are related to search pagination</p> -<div class="source"> -<pre> +<p>Search APIs (DSL, basic search, full-text search) support pagination and have optional limit and offset arguments. Following configs are related to search pagination</p> +<div class="source"><pre class="prettyprint"> # Default limit used when limit is not specified in API atlas.search.defaultlimit=100 @@ -330,55 +159,38 @@ atlas.search.maxlimit=10000 <div class="section"> <h3><a name="Notification_Configs"></a>Notification Configs</h3> <p>Refer <a class="externalLink" href="http://kafka.apache.org/documentation.html#configuration";>http://kafka.apache.org/documentation.html#configuration</a> for Kafka configuration. All Kafka configs should be prefixed with 'atlas.kafka.'</p> -<div class="source"> -<pre> -atlas.notification.embedded=true -atlas.kafka.data=${sys:atlas.home}/data/kafka -atlas.kafka.zookeeper.connect=localhost:9026 -atlas.kafka.bootstrap.servers=localhost:9027 -atlas.kafka.zookeeper.session.timeout.ms=400 -atlas.kafka.zookeeper.sync.time.ms=20 -atlas.kafka.auto.commit.interval.ms=1000 +<div class="source"><pre class="prettyprint"> +atlas.kafka.auto.commit.enable=false + +# Kafka servers. Example: localhost:6667 +atlas.kafka.bootstrap.servers= + atlas.kafka.hook.group.id=atlas -</pre></div> -<p>Note that Kafka group ids are specified for a specific topic. The Kafka group id configuration for entity notifications is 'atlas.kafka.entities.group.id'</p> -<div class="source"> -<pre> -atlas.kafka.entities.group.id=<consumer id> +# Zookeeper connect URL for Kafka. Example: localhost:2181 +atlas.kafka.zookeeper.connect= -</pre></div> -<p>These configuration parameters are useful for setting up Kafka topics via Atlas provided scripts, described in the <a href="./InstallationSteps.html">Installation Steps</a> page.</p> -<div class="source"> -<pre> -# Whether to create the topics automatically, default is true. -# Comma separated list of topics to be created, default is "ATLAS_HOOK,ATLAS_ENTITES" -atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES -# Number of replicas for the Atlas topics, default is 1. Increase for higher resilience to Kafka failures. -atlas.notification.replicas=1 -# Enable the below two properties if Kafka is running in Kerberized mode. -# Set this to the service principal representing the Kafka service -atlas.notification.kafka.service.principal=kafka/[email protected] -# Set this to the location of the keytab file for Kafka -#atlas.notification.kafka.keytab.location=/etc/security/keytabs/kafka.service.keytab +atlas.kafka.zookeeper.connection.timeout.ms=30000 +atlas.kafka.zookeeper.session.timeout.ms=60000 +atlas.kafka.zookeeper.sync.time.ms=20 -</pre></div> -<p>These configuration parameters are useful for saving messages in case there are issues in reaching Kafka for sending messages.</p> -<div class="source"> -<pre> -# Whether to save messages that failed to be sent to Kafka, default is true -atlas.notification.log.failed.messages=true -# If saving messages is enabled, the file name to save them to. This file will be created under the log directory of the hook's host component - like HiveServer2 -atlas.notification.failed.messages.filename=atlas_hook_failed_messages.log +# Setup the following configurations only in test deployments where Kafka is started within Atlas in embedded mode +# atlas.notification.embedded=true +# atlas.kafka.data=${sys:atlas.home}/data/kafka + +# Setup the following two properties if Kafka is running in Kerberized mode. +# atlas.notification.kafka.service.principal=kafka/[email protected] +# atlas.notification.kafka.keytab.location=/etc/security/keytabs/kafka.service.keytab </pre></div></div> <div class="section"> <h3><a name="Client_Configs"></a>Client Configs</h3> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> atlas.client.readTimeoutMSecs=60000 atlas.client.connectTimeoutMSecs=60000 -atlas.rest.address=<http/https>://<atlas-fqdn>:<atlas port> - default http://localhost:21000 + +# URL to access Atlas server. For example: http://localhost:21000 +atlas.rest.address= </pre></div></div> <div class="section"> @@ -386,32 +198,33 @@ atlas.rest.address=<http/https>://<atlas-fqdn>:<atlas port> - <div class="section"> <h4><a name="SSL_config"></a>SSL config</h4> <p>The following property is used to toggle the SSL feature.</p> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> atlas.enableTLS=false </pre></div></div> <div class="section"> <h3><a name="High_Availability_Properties"></a>High Availability Properties</h3> <p>The following properties describe High Availability related configuration options:</p> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> # Set the following property to true, to enable High Availability. Default = false. atlas.server.ha.enabled=true -# Define a unique set of strings to identify each instance that should run an Atlas Web Service instance as a comma separated list. +# Specify the list of Atlas instances atlas.server.ids=id1,id2 -# For each string defined above, define the host and port on which Atlas server binds to. +# For each instance defined above, define the host and port on which Atlas server listens. atlas.server.address.id1=host1.company.com:21000 atlas.server.address.id2=host2.company.com:31000 # Specify Zookeeper properties needed for HA. # Specify the list of services running Zookeeper servers as a comma separated list. atlas.server.ha.zookeeper.connect=zk1.company.com:2181,zk2.company.com:2181,zk3.company.com:2181 + # Specify how many times should connection try to be established with a Zookeeper cluster, in case of any connection issues. atlas.server.ha.zookeeper.num.retries=3 + # Specify how much time should the server wait before attempting connections to Zookeeper, in case of any connection issues. atlas.server.ha.zookeeper.retry.sleeptime.ms=1000 + # Specify how long a session to Zookeeper should last without inactiviy to be deemed as unreachable. atlas.server.ha.zookeeper.session.timeout.ms=20000 @@ -419,6 +232,7 @@ atlas.server.ha.zookeeper.session.timeout.ms=20000 # The format of these options is <scheme>:<identity>. For more information refer to http://zookeeper.apache.org/doc/r3.2.2/zookeeperProgrammers.html#sc_ZooKeeperAccessControl. # The 'acl' option allows to specify a scheme, identity pair to setup an ACL for. atlas.server.ha.zookeeper.acl=sasl:[email protected] + # The 'auth' option specifies the authentication that should be used for connecting to Zookeeper. atlas.server.ha.zookeeper.auth=sasl:[email protected] @@ -435,8 +249,7 @@ atlas.client.ha.sleep.interval.ms=5000 </pre></div></div> <div class="section"> <h3><a name="Server_Properties"></a>Server Properties</h3> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> # Set the following property to true, to enable the setup steps to run on each server start. Default = false. atlas.server.run.setup.on.start=false @@ -444,8 +257,7 @@ atlas.server.run.setup.on.start=false <div class="section"> <h3><a name="Performance_configuration_items"></a>Performance configuration items</h3> <p>The following properties can be used to tune performance of Atlas under specific circumstances:</p> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> # The number of times Atlas code tries to acquire a lock (to ensure consistency) while committing a transaction. # This should be related to the amount of concurrency expected to be supported by the server. For e.g. with retries set to 10, upto 100 threads can concurrently create types in the Atlas system. # If this is set to a low value (default is 3), concurrent operations might fail with a PermanentLockingException. @@ -470,36 +282,34 @@ atlas.webserver.queuesize=100 </pre></div></div> <div class="section"> <h4><a name="Recording_performance_metrics"></a>Recording performance metrics</h4> -<p>Atlas package should be built with '-P perf' to instrument atlas code to collect metrics. The metrics will be recorded in <atlas.log.dir>/metric.log, with one log line per API call. The metrics contain the number of times the instrumented methods are called and the total time spent in the instrumented method. Logging to metric.log is controlled through log4j configuration in atlas-log4j.xml. When the atlas code is instrumented, to disable logging to metric.log at runtime, set log level of METRICS logger to info level:</p> -<div class="source"> -<pre> -<logger name="METRICS" additivity="false"> - <level value="info"/> - <appender-ref ref="METRICS"/> -</logger> - +<p>To enable performance logs for various Atlas operations (like REST API calls, notification processing), setup the following in atlas-log4j.xml:</p> +<div class="source"><pre class="prettyprint"> + <appender name="perf_appender" class="org.apache.log4j.DailyRollingFileAppender"> + <param name="File" value="/var/log/atlas/atlas_perf.log"/> + <param name="datePattern" value="'.'yyyy-MM-dd"/> + <param name="append" value="true"/> + <layout class="org.apache.log4j.PatternLayout"> + <param name="ConversionPattern" value="%d|%t|%m%n"/> + </layout> + </appender> + + <logger name="org.apache.atlas.perf" additivity="false"> + <level value="debug"/> + <appender-ref ref="perf_appender"/> + </logger> + </pre></div></div> - </div> - </div> - + </div> + </div> <hr/> - <footer> - <div class="container"> - <div class="row span12">Copyright © 2015-2017 - <a href="http://www.apache.org";>Apache Software Foundation</a>. - All Rights Reserved. - + <div class="container"> + <div class="row"> +Copyright é 2018 The Apache Software Foundation, Licensed under the Apache License, Version 2.0. + </div> + <p id="poweredBy" class="pull-right"><a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /></a> +</p> </div> - - - <p id="poweredBy" class="pull-right"> - <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> - <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> - </a> - </p> - - </div> </footer> </body> </html>
http://git-wip-us.apache.org/repos/asf/atlas-website/blob/c3e5130b/EclipseSetup.html ---------------------------------------------------------------------- diff --git a/EclipseSetup.html b/EclipseSetup.html index 406804c..27d4912 100644 --- a/EclipseSetup.html +++ b/EclipseSetup.html @@ -1,220 +1,117 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia at 2017-08-09 - | Rendered using Apache Maven Fluido Skin 1.3.0 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/twiki/EclipseSetup.twiki at 2018-01-25 + | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20170809" /> + <meta name="Date-Revision-yyyymmdd" content="20180125" /> <meta http-equiv="Content-Language" content="en" /> <title>Apache Atlas – Tools required to build and run Apache Atlas on Eclipse</title> - <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.7.min.css" /> <link rel="stylesheet" href="./css/site.css" /> <link rel="stylesheet" href="./css/print.css" media="print" /> - - - <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script> - - - -<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> - - </head> - <body class="topBarEnabled"> - - - - - + <script type="text/javascript" src="./js/apache-maven-fluido-1.7.min.js"></script> + </head> + <body class="topBarEnabled"> <div id="topbar" class="navbar navbar-fixed-top "> <div class="navbar-inner"> - <div class="container" style="width: 68%;"><div class="nav-collapse"> - - - <ul class="nav"> - <li class="dropdown"> + <div class="container" style="width: 68%;"><div class="nav-collapse"> + <ul class="nav"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Atlas <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="index.html" title="About">About</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/ATLAS"; title="Wiki">Wiki</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/ATLAS"; title="News">News</a> -</li> - - <li> <a href="https://git-wip-us.apache.org/repos/asf/atlas.git"; title="Git">Git</a> -</li> - - <li> <a href="https://issues.apache.org/jira/browse/ATLAS"; title="Jira">Jira</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/ATLAS/PoweredBy"; title="Powered by">Powered by</a> -</li> - - <li> <a href="http://blogs.apache.org/atlas/"; title="Blog">Blog</a> -</li> - </ul> + <li><a href="index.html" title="About">About</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/ATLAS"; title="Wiki">Wiki</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/ATLAS"; title="News">News</a></li> + <li><a href="https://git-wip-us.apache.org/repos/asf/atlas.git"; title="Git">Git</a></li> + <li><a href="https://issues.apache.org/jira/browse/ATLAS"; title="Jira">Jira</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/ATLAS/PoweredBy"; title="Powered by">Powered by</a></li> + <li><a href="http://blogs.apache.org/atlas/"; title="Blog">Blog</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Project Information <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="project-info.html" title="Summary">Summary</a> -</li> - - <li> <a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a> -</li> - - <li> <a href="http://webchat.freenode.net?channels=apacheatlas&uio=d4"; title="IRC">IRC</a> -</li> - - <li> <a href="team-list.html" title="Team">Team</a> -</li> - - <li> <a href="issue-tracking.html" title="Issue Tracking">Issue Tracking</a> -</li> - - <li> <a href="source-repository.html" title="Source Repository">Source Repository</a> -</li> - - <li> <a href="license.html" title="License">License</a> -</li> - </ul> + <li><a href="project-info.html" title="Summary">Summary</a></li> + <li><a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a></li> + <li><a href="http://webchat.freenode.net?channels=apacheatlas&uio=d4"; title="IRC">IRC</a></li> + <li><a href="team-list.html" title="Team">Team</a></li> + <li><a href="issue-tracking.html" title="Issue Tracking">Issue Tracking</a></li> + <li><a href="source-repository.html" title="Source Repository">Source Repository</a></li> + <li><a href="license.html" title="License">License</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Releases <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="http://www.apache.org/dyn/closer.cgi/atlas/0.8.0-incubating/"; title="0.8-incubating">0.8-incubating</a> -</li> - - <li> <a href="http://archive.apache.org/dist/incubator/atlas/0.7.1-incubating/"; title="0.7.1-incubating">0.7.1-incubating</a> -</li> - - <li> <a href="http://archive.apache.org/dist/incubator/atlas/0.7.0-incubating/"; title="0.7-incubating">0.7-incubating</a> -</li> - - <li> <a href="http://archive.apache.org/dist/incubator/atlas/0.6.0-incubating/"; title="0.6-incubating">0.6-incubating</a> -</li> - - <li> <a href="http://archive.apache.org/dist/incubator/atlas/0.5.0-incubating/"; title="0.5-incubating">0.5-incubating</a> -</li> - </ul> + <li><a href="http://www.apache.org/dyn/closer.cgi/atlas/1.0.0-alpha/"; title="1.0.0-alpha">1.0.0-alpha</a></li> + <li><a href="http://www.apache.org/dyn/closer.cgi/atlas/0.8.1/"; title="0.8.1">0.8.1</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.8.0-incubating/"; title="0.8-incubating">0.8-incubating</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.7.1-incubating/"; title="0.7.1-incubating">0.7.1-incubating</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.7.0-incubating/"; title="0.7-incubating">0.7-incubating</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.6.0-incubating/"; title="0.6-incubating">0.6-incubating</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.5.0-incubating/"; title="0.5-incubating">0.5-incubating</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentation <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="0.8.0-incubating/index.html" title="0.8-incubating">0.8-incubating</a> -</li> - - <li> <a href="0.7.1-incubating/index.html" title="0.7.1-incubating">0.7.1-incubating</a> -</li> - - <li> <a href="0.7.0-incubating/index.html" title="0.7-incubating">0.7-incubating</a> -</li> - - <li> <a href="0.6.0-incubating/index.html" title="0.6-incubating">0.6-incubating</a> -</li> - - <li> <a href="0.5.0-incubating/index.html" title="0.5-incubating">0.5-incubating</a> -</li> - </ul> + <li><a href="1.0.0-alpha/index.html" title="1.0.0-alpha">1.0.0-alpha</a></li> + <li><a href="0.8.1/index.html" title="0.8.1">0.8.1</a></li> + <li><a href="0.8.0-incubating/index.html" title="0.8-incubating">0.8-incubating</a></li> + <li><a href="0.7.1-incubating/index.html" title="0.7.1-incubating">0.7.1-incubating</a></li> + <li><a href="0.7.0-incubating/index.html" title="0.7-incubating">0.7-incubating</a></li> + <li><a href="0.6.0-incubating/index.html" title="0.6-incubating">0.6-incubating</a></li> + <li><a href="0.5.0-incubating/index.html" title="0.5-incubating">0.5-incubating</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">ASF <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="http://www.apache.org/foundation/how-it-works.html"; title="How Apache Works">How Apache Works</a> -</li> - - <li> <a href="http://www.apache.org/foundation/"; title="Foundation">Foundation</a> -</li> - - <li> <a href="http://www.apache.org/foundation/sponsorship.html"; title="Sponsoring Apache">Sponsoring Apache</a> -</li> - - <li> <a href="http://www.apache.org/foundation/thanks.html"; title="Thanks">Thanks</a> -</li> - </ul> + <li><a href="http://www.apache.org/foundation/how-it-works.html"; title="How Apache Works">How Apache Works</a></li> + <li><a href="http://www.apache.org/foundation/"; title="Foundation">Foundation</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html"; title="Sponsoring Apache">Sponsoring Apache</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html"; title="Thanks">Thanks</a></li> + </ul> </li> - </ul> - - <form id="search-form" action="http://www.google.com/search"; method="get" class="navbar-search pull-right" > - + </ul> +<form id="search-form" action="https://www.google.com/search"; method="get" class="navbar-search pull-right" > <input value="http://atlas.apache.org"; name="sitesearch" type="hidden"/> <input class="search-query" name="q" id="query" type="text" /> </form> -<script type="text/javascript" src="http://www.google.com/coop/cse/brand?form=search-form";></script> - - - - - - <iframe src="http://www.facebook.com/plugins/like.php?href=http://atlas.apache.org/atlas-docs&send=false&layout=button_count&show-faces=false&action=like&colorscheme=dark"; +<script type="text/javascript">asyncJs( 'https://cse.google.com/brand?form=search-form' )</script> + <iframe src="https://www.facebook.com/plugins/like.php?href=http://atlas.apache.org/atlas-docs&send=false&layout=button_count&show-faces=false&action=like&colorscheme=dark"; scrolling="no" frameborder="0" - style="border:none; width:80px; height:20px; margin-top: 10px;" class="pull-right" ></iframe> - - <script type="text/javascript" src="https://apis.google.com/js/plusone.js";></script> - - <ul class="nav pull-right"><li style="margin-top: 10px;"> - + style="border:none; width:100px; height:20px; margin-top: 10px;" class="pull-right" ></iframe> + <script type="text/javascript">asyncJs( 'https://apis.google.com/js/plusone.js' )</script> + <ul class="nav pull-right"><li style="margin-top: 10px;"> <div class="g-plusone" data-href="http://atlas.apache.org/atlas-docs"; data-size="medium" width="60px" align="right" ></div> - - </li></ul> - - - </div> - + </li></ul> + </div> </div> </div> </div> - - <div class="container"> - <div id="banner"> - <div class="pull-left"> - <a href=".." id="bannerLeft"> - <img src="images/atlas-logo.png" alt="Apache Atlas" width="200px" height="45px"/> - </a> - </div> - <div class="pull-right"> </div> + <div class="container"> + <div id="banner"> + <div class="pull-left"><a href=".." id="bannerLeft"><img src="images/atlas-logo.png" alt="Apache Atlas" width="200px" height="45px"/></a></div> + <div class="pull-right"></div> <div class="clear"><hr/></div> </div> <div id="breadcrumbs"> <ul class="breadcrumb"> - - - <li class=""> - <a href="http://www.apache.org"; class="externalLink" title="Apache"> - Apache</a> - </li> - <li class="divider ">/</li> - <li class=""> - <a href="index.html" title="Atlas"> - Atlas</a> - </li> - <li class="divider ">/</li> - <li class="">Tools required to build and run Apache Atlas on Eclipse</li> - - - - <li id="publishDate" class="pull-right">Last Published: 2017-08-09</li> <li class="divider pull-right">|</li> - <li id="projectVersion" class="pull-right">Version: 0.9-SNAPSHOT</li> - - </ul> + <li class=""><a href="http://www.apache.org"; class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li> + <li class=""><a href="index.html" title="Atlas">Atlas</a><span class="divider">/</span></li> + <li class="active ">Tools required to build and run Apache Atlas on Eclipse</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-01-25</li> + <li id="projectVersion" class="pull-right">Version: 1.0.0-alpha</li> + </ul> </div> - - - <div id="bodyColumn" > - - <div class="section"> +<div class="section"> <h2><a name="Tools_required_to_build_and_run_Apache_Atlas_on_Eclipse"></a>Tools required to build and run Apache Atlas on Eclipse</h2> <p>These instructions are provided as-is. They worked at a point in time; other variants of software may work. These instructions may become stale if the build dependencies change.</p> <p>They have been shown to work on 19th of December 2016.</p> @@ -263,8 +160,7 @@ <li>In Window - Preferences -Maven - Errors/Warnings, set Plugin execution not covered by lifecycle configuration to Warning</li></ul><b>Import Atlas maven projects into Eclipse:</b> <p>a. File - Import - Maven - Existing Maven Projects b. Browse to your Atlas folder c. Uncheck the root project and non-Java projects such as dashboardv2, docs and distro, then click Finish</p> <p>On the Mac, the Maven import fails with message</p> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> "Cannot complete the install because one or more required items could not be found. Software being installed: Maven Integration for AJDT (Optional) 0.14.0.201506231302 (org.maven.ide.eclipse.ajdt.feature.feature.group 0.14.0.201506231302) Missing requirement: Maven Integration for AJDT (Optional) 0.14.0.201506231302 (org.maven.ide.eclipse.ajdt.feature.feature.group 0.14.0.201506231302) requires 'org.eclipse.ajdt.core 1.5.0' but it could not be found". </pre></div> @@ -284,21 +180,19 @@ <p>You will need to change some of these scripts to point to your installation targets.</p> <ul> <li>Run this script to setup your command line build environment</li></ul> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> #!/bin/bash # export JAVA_HOME=/Library/Java/JavaVirtualMachines/macosxx6480sr3fp10hybrid-20160719_01-sdk export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home export M2_HOME=/Applications/apache-maven-3.3.9 # Git is installed in the system path export PYTHON_HOME='/Applications/Python 2.7' export PATH=$PYTHON_HOME:$M2_HOME/bin:$JAVA_HOME/bin:$PATH -export MAVEN_OPTS="-Xmx1536m -Drat.numUnapprovedLicenses=100 -XX:MaxPermSize=256m" +export MAVEN_OPTS="-Xmx1536m -Drat.numUnapprovedLicenses=100" </pre></div> <p></p> <ul> <li>If you do not want to set Java 8 as your system java, you can use this bash script to setup the environment and run Eclipse (which you can drop in Applications and rename to neon).</li></ul> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> #!/bin/bash export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home export M2_HOME=/Applications/apache-maven-3.3.9 @@ -307,27 +201,17 @@ export PYTHON_HOME='/Applications/Python 2.7' export PATH=$PYTHON_HOME:$M2_HOME/bin:$JAVA_HOME/bin:$PATH/Applications/neon.app/Contents/MacOS/eclipse </pre></div></div> - </div> - </div> - + </div> + </div> <hr/> - <footer> - <div class="container"> - <div class="row span12">Copyright © 2015-2017 - <a href="http://www.apache.org";>Apache Software Foundation</a>. - All Rights Reserved. - + <div class="container"> + <div class="row"> +Copyright é 2018 The Apache Software Foundation, Licensed under the Apache License, Version 2.0. + </div> + <p id="poweredBy" class="pull-right"><a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /></a> +</p> </div> - - - <p id="poweredBy" class="pull-right"> - <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> - <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> - </a> - </p> - - </div> </footer> </body> </html> http://git-wip-us.apache.org/repos/asf/atlas-website/blob/c3e5130b/Export-API.html ---------------------------------------------------------------------- diff --git a/Export-API.html b/Export-API.html index d8c3917..0d6fd54 100644 --- a/Export-API.html +++ b/Export-API.html @@ -1,220 +1,117 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia at 2017-08-09 - | Rendered using Apache Maven Fluido Skin 1.3.0 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/twiki/Export-API.twiki at 2018-01-25 + | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20170809" /> + <meta name="Date-Revision-yyyymmdd" content="20180125" /> <meta http-equiv="Content-Language" content="en" /> <title>Apache Atlas – Export API</title> - <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.7.min.css" /> <link rel="stylesheet" href="./css/site.css" /> <link rel="stylesheet" href="./css/print.css" media="print" /> - - - <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script> - - - -<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> - - </head> - <body class="topBarEnabled"> - - - - - + <script type="text/javascript" src="./js/apache-maven-fluido-1.7.min.js"></script> + </head> + <body class="topBarEnabled"> <div id="topbar" class="navbar navbar-fixed-top "> <div class="navbar-inner"> - <div class="container" style="width: 68%;"><div class="nav-collapse"> - - - <ul class="nav"> - <li class="dropdown"> + <div class="container" style="width: 68%;"><div class="nav-collapse"> + <ul class="nav"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Atlas <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="index.html" title="About">About</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/ATLAS"; title="Wiki">Wiki</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/ATLAS"; title="News">News</a> -</li> - - <li> <a href="https://git-wip-us.apache.org/repos/asf/atlas.git"; title="Git">Git</a> -</li> - - <li> <a href="https://issues.apache.org/jira/browse/ATLAS"; title="Jira">Jira</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/ATLAS/PoweredBy"; title="Powered by">Powered by</a> -</li> - - <li> <a href="http://blogs.apache.org/atlas/"; title="Blog">Blog</a> -</li> - </ul> + <li><a href="index.html" title="About">About</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/ATLAS"; title="Wiki">Wiki</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/ATLAS"; title="News">News</a></li> + <li><a href="https://git-wip-us.apache.org/repos/asf/atlas.git"; title="Git">Git</a></li> + <li><a href="https://issues.apache.org/jira/browse/ATLAS"; title="Jira">Jira</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/ATLAS/PoweredBy"; title="Powered by">Powered by</a></li> + <li><a href="http://blogs.apache.org/atlas/"; title="Blog">Blog</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Project Information <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="project-info.html" title="Summary">Summary</a> -</li> - - <li> <a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a> -</li> - - <li> <a href="http://webchat.freenode.net?channels=apacheatlas&uio=d4"; title="IRC">IRC</a> -</li> - - <li> <a href="team-list.html" title="Team">Team</a> -</li> - - <li> <a href="issue-tracking.html" title="Issue Tracking">Issue Tracking</a> -</li> - - <li> <a href="source-repository.html" title="Source Repository">Source Repository</a> -</li> - - <li> <a href="license.html" title="License">License</a> -</li> - </ul> + <li><a href="project-info.html" title="Summary">Summary</a></li> + <li><a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a></li> + <li><a href="http://webchat.freenode.net?channels=apacheatlas&uio=d4"; title="IRC">IRC</a></li> + <li><a href="team-list.html" title="Team">Team</a></li> + <li><a href="issue-tracking.html" title="Issue Tracking">Issue Tracking</a></li> + <li><a href="source-repository.html" title="Source Repository">Source Repository</a></li> + <li><a href="license.html" title="License">License</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Releases <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="http://www.apache.org/dyn/closer.cgi/atlas/0.8.0-incubating/"; title="0.8-incubating">0.8-incubating</a> -</li> - - <li> <a href="http://archive.apache.org/dist/incubator/atlas/0.7.1-incubating/"; title="0.7.1-incubating">0.7.1-incubating</a> -</li> - - <li> <a href="http://archive.apache.org/dist/incubator/atlas/0.7.0-incubating/"; title="0.7-incubating">0.7-incubating</a> -</li> - - <li> <a href="http://archive.apache.org/dist/incubator/atlas/0.6.0-incubating/"; title="0.6-incubating">0.6-incubating</a> -</li> - - <li> <a href="http://archive.apache.org/dist/incubator/atlas/0.5.0-incubating/"; title="0.5-incubating">0.5-incubating</a> -</li> - </ul> + <li><a href="http://www.apache.org/dyn/closer.cgi/atlas/1.0.0-alpha/"; title="1.0.0-alpha">1.0.0-alpha</a></li> + <li><a href="http://www.apache.org/dyn/closer.cgi/atlas/0.8.1/"; title="0.8.1">0.8.1</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.8.0-incubating/"; title="0.8-incubating">0.8-incubating</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.7.1-incubating/"; title="0.7.1-incubating">0.7.1-incubating</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.7.0-incubating/"; title="0.7-incubating">0.7-incubating</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.6.0-incubating/"; title="0.6-incubating">0.6-incubating</a></li> + <li><a href="http://archive.apache.org/dist/incubator/atlas/0.5.0-incubating/"; title="0.5-incubating">0.5-incubating</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentation <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="0.8.0-incubating/index.html" title="0.8-incubating">0.8-incubating</a> -</li> - - <li> <a href="0.7.1-incubating/index.html" title="0.7.1-incubating">0.7.1-incubating</a> -</li> - - <li> <a href="0.7.0-incubating/index.html" title="0.7-incubating">0.7-incubating</a> -</li> - - <li> <a href="0.6.0-incubating/index.html" title="0.6-incubating">0.6-incubating</a> -</li> - - <li> <a href="0.5.0-incubating/index.html" title="0.5-incubating">0.5-incubating</a> -</li> - </ul> + <li><a href="1.0.0-alpha/index.html" title="1.0.0-alpha">1.0.0-alpha</a></li> + <li><a href="0.8.1/index.html" title="0.8.1">0.8.1</a></li> + <li><a href="0.8.0-incubating/index.html" title="0.8-incubating">0.8-incubating</a></li> + <li><a href="0.7.1-incubating/index.html" title="0.7.1-incubating">0.7.1-incubating</a></li> + <li><a href="0.7.0-incubating/index.html" title="0.7-incubating">0.7-incubating</a></li> + <li><a href="0.6.0-incubating/index.html" title="0.6-incubating">0.6-incubating</a></li> + <li><a href="0.5.0-incubating/index.html" title="0.5-incubating">0.5-incubating</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">ASF <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="http://www.apache.org/foundation/how-it-works.html"; title="How Apache Works">How Apache Works</a> -</li> - - <li> <a href="http://www.apache.org/foundation/"; title="Foundation">Foundation</a> -</li> - - <li> <a href="http://www.apache.org/foundation/sponsorship.html"; title="Sponsoring Apache">Sponsoring Apache</a> -</li> - - <li> <a href="http://www.apache.org/foundation/thanks.html"; title="Thanks">Thanks</a> -</li> - </ul> + <li><a href="http://www.apache.org/foundation/how-it-works.html"; title="How Apache Works">How Apache Works</a></li> + <li><a href="http://www.apache.org/foundation/"; title="Foundation">Foundation</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html"; title="Sponsoring Apache">Sponsoring Apache</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html"; title="Thanks">Thanks</a></li> + </ul> </li> - </ul> - - <form id="search-form" action="http://www.google.com/search"; method="get" class="navbar-search pull-right" > - + </ul> +<form id="search-form" action="https://www.google.com/search"; method="get" class="navbar-search pull-right" > <input value="http://atlas.apache.org"; name="sitesearch" type="hidden"/> <input class="search-query" name="q" id="query" type="text" /> </form> -<script type="text/javascript" src="http://www.google.com/coop/cse/brand?form=search-form";></script> - - - - - - <iframe src="http://www.facebook.com/plugins/like.php?href=http://atlas.apache.org/atlas-docs&send=false&layout=button_count&show-faces=false&action=like&colorscheme=dark"; +<script type="text/javascript">asyncJs( 'https://cse.google.com/brand?form=search-form' )</script> + <iframe src="https://www.facebook.com/plugins/like.php?href=http://atlas.apache.org/atlas-docs&send=false&layout=button_count&show-faces=false&action=like&colorscheme=dark"; scrolling="no" frameborder="0" - style="border:none; width:80px; height:20px; margin-top: 10px;" class="pull-right" ></iframe> - - <script type="text/javascript" src="https://apis.google.com/js/plusone.js";></script> - - <ul class="nav pull-right"><li style="margin-top: 10px;"> - + style="border:none; width:100px; height:20px; margin-top: 10px;" class="pull-right" ></iframe> + <script type="text/javascript">asyncJs( 'https://apis.google.com/js/plusone.js' )</script> + <ul class="nav pull-right"><li style="margin-top: 10px;"> <div class="g-plusone" data-href="http://atlas.apache.org/atlas-docs"; data-size="medium" width="60px" align="right" ></div> - - </li></ul> - - - </div> - + </li></ul> + </div> </div> </div> </div> - - <div class="container"> - <div id="banner"> - <div class="pull-left"> - <a href=".." id="bannerLeft"> - <img src="images/atlas-logo.png" alt="Apache Atlas" width="200px" height="45px"/> - </a> - </div> - <div class="pull-right"> </div> + <div class="container"> + <div id="banner"> + <div class="pull-left"><a href=".." id="bannerLeft"><img src="images/atlas-logo.png" alt="Apache Atlas" width="200px" height="45px"/></a></div> + <div class="pull-right"></div> <div class="clear"><hr/></div> </div> <div id="breadcrumbs"> <ul class="breadcrumb"> - - - <li class=""> - <a href="http://www.apache.org"; class="externalLink" title="Apache"> - Apache</a> - </li> - <li class="divider ">/</li> - <li class=""> - <a href="index.html" title="Atlas"> - Atlas</a> - </li> - <li class="divider ">/</li> - <li class="">Export API</li> - - - - <li id="publishDate" class="pull-right">Last Published: 2017-08-09</li> <li class="divider pull-right">|</li> - <li id="projectVersion" class="pull-right">Version: 0.9-SNAPSHOT</li> - - </ul> + <li class=""><a href="http://www.apache.org"; class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li> + <li class=""><a href="index.html" title="Atlas">Atlas</a><span class="divider">/</span></li> + <li class="active ">Export API</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-01-25</li> + <li id="projectVersion" class="pull-right">Version: 1.0.0-alpha</li> + </ul> </div> - - - <div id="bodyColumn" > - - <div class="section"> +<div class="section"> <h2><a name="Export_API"></a>Export API</h2> <p>The general approach is:</p> <ul> @@ -251,8 +148,7 @@ <tr class="a"> <td><i>Notes</i></td> <td>Consumer could choose to consume the output of the API by programmatically using <i>java.io.ByteOutputStream</i> or by manually, save the contents of the stream to a file on the disk.</td></tr></table><b><i>Method Signature</i></b> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> @POST @Path("/export") @Consumes("application/json;charset=UTF-8") @@ -294,8 +190,7 @@ <div class="section"> <h4><a name="Examples"></a>Examples</h4> <p>The <i>AtlasExportRequest</i> below shows filters that attempt to export 2 databases in cluster cl1:</p> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> { "itemsToExport": [ { "typeName": "hive_db", "uniqueAttributes": { "qualifiedName": "accounts@cl1" } }, @@ -305,8 +200,7 @@ </pre></div> <p>The <i>AtlasExportRequest</i> below specifies the <i>fetchType</i> as <i>FULL</i>. The <i>matchType</i> option will fetch <i>accounts@cl1</i>.</p> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> { "itemsToExport": [ { "typeName": "hive_db", "uniqueAttributes": { "qualifiedName": "accounts@" } }, @@ -319,8 +213,7 @@ </pre></div> <p>The <i>AtlasExportRequest</i> below specifies the <i>fetchType</i> as <i>connected</i>. The <i>matchType</i> option will fetch <i>accountsReceivable</i>, <i>accountsPayable</i>, etc present in the database.</p> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> { "itemsToExport": [ { "typeName": "hive_db", "uniqueAttributes": { "name": "accounts" } }, @@ -334,8 +227,7 @@ </pre></div> <p>Below is the <i>AtlasExportResult</i> JSON for the export of the <i>Sales</i> DB present in the <i>QuickStart</i>.</p> <p>The <i>metrics</i> contains the number of types and entities exported as part of the operation.</p> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> { "clientIpAddress": "10.0.2.15", "hostName": "10.0.2.2", @@ -375,8 +267,7 @@ <div class="section"> <h4><a name="CURL_Calls"></a>CURL Calls</h4> <p>Below are sample CURL calls that demonstrate Export of <i>QuickStart</i> database.</p> -<div class="source"> -<pre> +<div class="source"><pre class="prettyprint"> curl -X POST -u adminuser:password -H "Content-Type: application/json" -H "Cache-Control: no-cache" -d '{ "itemsToExport": [ { "typeName": "DB", "uniqueAttributes": { "name": "Sales" } @@ -388,27 +279,17 @@ curl -X POST -u adminuser:password -H "Content-Type: application/json" }' "http://localhost:21000/api/atlas/admin/export" > quickStartDB.zip </pre></div></div> - </div> - </div> - + </div> + </div> <hr/> - <footer> - <div class="container"> - <div class="row span12">Copyright © 2015-2017 - <a href="http://www.apache.org";>Apache Software Foundation</a>. - All Rights Reserved. - + <div class="container"> + <div class="row"> +Copyright é 2018 The Apache Software Foundation, Licensed under the Apache License, Version 2.0. + </div> + <p id="poweredBy" class="pull-right"><a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /></a> +</p> </div> - - - <p id="poweredBy" class="pull-right"> - <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> - <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> - </a> - </p> - - </div> </footer> </body> </html>
