This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/amoro-site.git
The following commit(s) were added to refs/heads/asf-site by this push: new aeca13f deploy: 335193651ea99a844ec6933fd175b5cc92ff11df aeca13f is described below commit aeca13fd8f3fe54ce6acfcfc31ab4168b97ba990 Author: zhoujinsong <zhoujins...@users.noreply.github.com> AuthorDate: Wed Jan 22 02:23:46 2025 +0000 deploy: 335193651ea99a844ec6933fd175b5cc92ff11df --- output/docs/latest/catalogs/index.html | 2 +- .../latest/deployment-on-kubernetes/index.html | 14 ++++---- output/docs/latest/deployment/index.html | 41 ++++++++-------------- output/docs/latest/search.json | 2 +- 4 files changed, 24 insertions(+), 35 deletions(-) diff --git a/output/docs/latest/catalogs/index.html b/output/docs/latest/catalogs/index.html index e2440c2..f04f22a 100644 --- a/output/docs/latest/catalogs/index.html +++ b/output/docs/latest/catalogs/index.html @@ -595,7 +595,7 @@ namespace for <code>table</code> and <code>database</code>. Typically, a catalog is associated with a specific type of data source or cluster. In Flink, Spark and Trino, the multi-catalog feature can be used to support SQL across data sources, such as:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-SQL" data-lang="SQL"><span style="display:flex;"><span><span style="color:#66d9ef">SELECT</span> <span style="color:#66d9ef">c</span>.ID, <span style="color:#66d9ef">c</span>.NAME, <span style="color:#66d9ef">c</span>.AGE, o.AMOUNT -</span></span><span style="display:flex;"><span><span style="color:#66d9ef">FROM</span> MYSQL.ONLINE.CUSTOMERS <span style="color:#66d9ef">c</span> <span style="color:#66d9ef">JOIN</span> HIVE.OFFLINE.ORDERS o +</span></span><span style="display:flex;"><span><span style="color:#66d9ef">FROM</span> <span style="color:#960050;background-color:#1e0010">${</span>CATALOG_A<span style="color:#960050;background-color:#1e0010">}</span>.ONLINE.CUSTOMERS <span style="color:#66d9ef">c</span> <span style="color:#66d9ef">JOIN</span> <span style="color:#960050;background-color:#1e0010">${</span>CATALOG_B<span style="color:#960050;background-color:#1e0010">}</span>.OFFLINE.ORDERS o </span></span><span style="display:flex;"><span><span style="color:#66d9ef">ON</span> (<span style="color:#66d9ef">c</span>.ID <span style="color:#f92672">=</span> o.CUSTOMER_ID) </span></span></code></pre></div><p>In the past, data lakes were managed using the Hive Metastore (HMS) to handle metadata. Unfortunately, HMS does not support multi-catalog, which limits the capabilities of engines on the data lake. For example, some users may want to use Spark to perform federated computation across different diff --git a/output/docs/latest/deployment-on-kubernetes/index.html b/output/docs/latest/deployment-on-kubernetes/index.html index fb8889c..5c5a2a3 100644 --- a/output/docs/latest/deployment-on-kubernetes/index.html +++ b/output/docs/latest/deployment-on-kubernetes/index.html @@ -652,15 +652,15 @@ </span></span><span style="display:flex;"><span> <span style="color:#f92672">ingressClassName</span>: <span style="color:#e6db74">"nginx"</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">hostname</span>: <span style="color:#ae81ff">minikube.amoro.com</span> </span></span></code></pre></div><h3 id="configure-the-database">Configure the database.</h3> -<p>AMS default is to use Derby database for storage. When the pod is destroyed, the data will also disappear. -In production environments, we recommend using MySQL as the storage for system data.</p> +<p>AMS uses embedded <a href="https://db.apache.org/derby/">Apache Derby</a> as its backend storage by default. +In production environments, we recommend using a RDBMS(Relational Database Management System) with higher availability guarantees as the storage for system data, you can ref to <a href="../deployment/#configure-system-database">Database Configuration</a> for more detail.</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#f92672">amoroConf</span>: </span></span><span style="display:flex;"><span> <span style="color:#f92672">database</span>: -</span></span><span style="display:flex;"><span> <span style="color:#f92672">type</span>: <span style="color:#ae81ff">mysql</span> -</span></span><span style="display:flex;"><span> <span style="color:#f92672">driver</span>: <span style="color:#ae81ff">com.mysql.cj.jdbc.Driver</span> -</span></span><span style="display:flex;"><span> <span style="color:#f92672">url</span>: <span style="color:#ae81ff"><jdbc-uri></span> -</span></span><span style="display:flex;"><span> <span style="color:#f92672">username</span>: <span style="color:#ae81ff"><mysql-user></span> -</span></span><span style="display:flex;"><span> <span style="color:#f92672">password</span>: <span style="color:#ae81ff"><mysql-password></span> +</span></span><span style="display:flex;"><span> <span style="color:#f92672">type</span>: <span style="color:#ae81ff">${your_database_type}</span> +</span></span><span style="display:flex;"><span> <span style="color:#f92672">driver</span>: <span style="color:#ae81ff">${your_database_driver}</span> +</span></span><span style="display:flex;"><span> <span style="color:#f92672">url</span>: <span style="color:#ae81ff">${your_jdbc_url}</span> +</span></span><span style="display:flex;"><span> <span style="color:#f92672">username</span>: <span style="color:#ae81ff">${your_username}</span> +</span></span><span style="display:flex;"><span> <span style="color:#f92672">password</span>: <span style="color:#ae81ff">${your_password}</span> </span></span></code></pre></div><h3 id="configure-the-images">Configure the Images</h3> <p>Helm charts deploy images by default using the latest tag. If you need to modify the image address, such as using a private repository or building your own image</p> diff --git a/output/docs/latest/deployment/index.html b/output/docs/latest/deployment/index.html index 7d6db08..1438ff5 100644 --- a/output/docs/latest/deployment/index.html +++ b/output/docs/latest/deployment/index.html @@ -594,8 +594,7 @@ <h2 id="system-requirements">System requirements</h2> <ul> <li>Java 8 is required.</li> -<li>Optional: MySQL 5.5 or higher</li> -<li>Optional: PostgreSQL 14.x or higher</li> +<li>Optional: A RDBMS (PostgreSQL 14.x or higher, MySQL 5.5 or higher)</li> <li>Optional: ZooKeeper 3.4.x or higher</li> </ul> <h2 id="download-the-distribution">Download the distribution</h2> @@ -647,31 +646,21 @@ Unzip it to create the amoro-x.y.z directory in the same directory, and then go Make sure the port is not used before configuring it. </div> <h3 id="configure-system-database">Configure system database</h3> -<p>You can use MySQL/PostgreSQL as the system database instead of the default Derby.</p> +<p>AMS uses embedded <a href="https://db.apache.org/derby/">Apache Derby</a> as the backend storage by default, so you can use <code>Derby</code> directly without any additional configuration.</p> +<p>You can also configure a relational backend storage as you needed.</p> +<blockquote> <p>If you would like to use MySQL as the system database, you need to manually download the <a href="https://repo1.maven.org/maven2/com/mysql/mysql-connector-j/8.1.0/mysql-connector-j-8.1.0.jar">MySQL JDBC Connector</a> -and move it into the <code>{AMORO_HOME}/lib/</code> directory. You can use the following command to complete these operations:</p> -<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>$ cd <span style="color:#e6db74">${</span>AMORO_HOME<span style="color:#e6db74">}</span> -</span></span><span style="display:flex;"><span>$ MYSQL_JDBC_DRIVER_VERSION<span style="color:#f92672">=</span>8.0.30 -</span></span><span style="display:flex;"><span>$ wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/<span style="color:#e6db74">${</span>MYSQL_JDBC_DRIVER_VERSION<span style="color:#e6db74">}</span>/mysql-connector-java-<span style="color:#e6db74">${</span>MYSQL_JDBC_DRIVER_VERSION<span style="color:#e6db74">}</span>.jar -</span></span><span style="display:flex;"><span>$ mv mysql-connector-java-<span style="color:#e6db74">${</span>MYSQL_JDBC_DRIVER_VERSION<span style="color:#e6db74">}</span>.jar lib -</span></span></code></pre></div><p>Create an empty database in MySQL/PostgreSQL, then AMS will automatically create tables in this MySQL/PostgreSQL database when it first started.</p> -<p>One thing you need to do is Adding MySQL/PostgreSQL configuration under <code>config.yaml</code> of Ams:</p> -<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#75715e"># MySQL</span> -</span></span><span style="display:flex;"><span><span style="color:#f92672">ams</span>: +and move it into the <code>${AMORO_HOME}/lib/</code> directory.</p> +</blockquote> +<p>You need to create an empty database in the RDBMS before to start the server, then AMS will automatically create tables in the database when it first started.</p> +<p>One thing you need to do is adding configuration under <code>config.yaml</code> of Ams:</p> +<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#f92672">ams</span>: </span></span><span style="display:flex;"><span> <span style="color:#f92672">database</span>: -</span></span><span style="display:flex;"><span> <span style="color:#f92672">type</span>: <span style="color:#ae81ff">mysql</span> -</span></span><span style="display:flex;"><span> <span style="color:#f92672">jdbc-driver-class</span>: <span style="color:#ae81ff">com.mysql.cj.jdbc.Driver</span> -</span></span><span style="display:flex;"><span> <span style="color:#f92672">url</span>: <span style="color:#ae81ff">jdbc:mysql://127.0.0.1:3306/amoro?useUnicode=true&characterEncoding=UTF8&autoReconnect=true&useAffectedRows=true&allowPublicKeyRetrieval=true&useSSL=false</span> -</span></span><span style="display:flex;"><span> <span style="color:#f92672">username</span>: <span style="color:#ae81ff">root</span> -</span></span><span style="display:flex;"><span> <span style="color:#f92672">password</span>: <span style="color:#ae81ff">root</span> -</span></span><span style="display:flex;"><span><span style="color:#75715e"># PostgreSQL</span> -</span></span><span style="display:flex;"><span><span style="color:#75715e">#ams:</span> -</span></span><span style="display:flex;"><span><span style="color:#75715e"># database:</span> -</span></span><span style="display:flex;"><span><span style="color:#75715e"># type: postgres</span> -</span></span><span style="display:flex;"><span><span style="color:#75715e"># jdbc-driver-class: org.postgresql.Driver</span> -</span></span><span style="display:flex;"><span><span style="color:#75715e"># url: jdbc:postgresql://127.0.0.1:5432/amoro</span> -</span></span><span style="display:flex;"><span><span style="color:#75715e"># username: user</span> -</span></span><span style="display:flex;"><span><span style="color:#75715e"># password: passwd</span> +</span></span><span style="display:flex;"><span> <span style="color:#f92672">type</span>: <span style="color:#ae81ff">${database_type}</span> <span style="color:#75715e"># postgres or mysql</span> +</span></span><span style="display:flex;"><span> <span style="color:#f92672">jdbc-driver-class</span>: <span style="color:#ae81ff">${your_driver_name}</span> +</span></span><span style="display:flex;"><span> <span style="color:#f92672">url</span>: <span style="color:#ae81ff">${your_jdbc_url}</span> +</span></span><span style="display:flex;"><span> <span style="color:#f92672">username</span>: <span style="color:#ae81ff">${your_username}</span> +</span></span><span style="display:flex;"><span> <span style="color:#f92672">password</span>: <span style="color:#ae81ff">${your_password}</span> </span></span></code></pre></div><h3 id="configure-high-availability">Configure high availability</h3> <p>To improve stability, AMS supports a one-master-multi-backup HA mode. Zookeeper is used to implement leader election, and the AMS cluster name and Zookeeper address are specified. The AMS cluster name is used to bind different AMS clusters @@ -851,7 +840,7 @@ successful. The default username and password for login are both “admin&rd </span></span><span style="display:flex;"><span>$ bin/ams.sh stop </span></span></code></pre></div><h2 id="upgrade-ams">Upgrade AMS</h2> <h3 id="upgrade-system-databases">Upgrade system databases</h3> -<p>You can find all the upgrade SQL scripts under <code>{AMORO_HOME}/conf/mysql/</code> with name pattern <code>upgrade-a.b.c-to-x.y.z.sql</code>. +<p>You can find all the upgrade SQL scripts under <code>${AMORO_HOME}/conf/${db_type}/</code> with name pattern <code>upgrade-a.b.c-to-x.y.z.sql</code>. Execute the upgrade SQL scripts one by one to your system database based on your starting and target versions.</p> <h3 id="replace-all-libs-and-plugins">Replace all libs and plugins</h3> <p>Replace all contents in the original <code>{AMORO_HOME}/lib</code> directory with the contents in the lib directory of the new installation package. diff --git a/output/docs/latest/search.json b/output/docs/latest/search.json index 35448f1..87267bf 100644 --- a/output/docs/latest/search.json +++ b/output/docs/latest/search.json @@ -1 +1 @@ -[{"categories":null,"content":"Amoro Docs This directory contains the documentation content of Amoro.\nThe documentation is written in Markdown format. The images referenced in the documentation are saved in the images folder. Style Proper nouns should start with a capital letter, like Hadoop、Hive、Iceberg、Amoro ","description":"","title":"","uri":"https://amoro.apache.org/docs/latest/readme/"},{"categories":null,"content":" Catalogs Introduce multi-catalog A catalog is a metadata namespa [...] \ No newline at end of file +[{"categories":null,"content":"Amoro Docs This directory contains the documentation content of Amoro.\nThe documentation is written in Markdown format. The images referenced in the documentation are saved in the images folder. Style Proper nouns should start with a capital letter, like Hadoop、Hive、Iceberg、Amoro ","description":"","title":"","uri":"https://amoro.apache.org/docs/latest/readme/"},{"categories":null,"content":" Catalogs Introduce multi-catalog A catalog is a metadata namespa [...] \ No newline at end of file