CHUKWA-786. Update documentation to reflect 0.7 release. (Eric Yang)
Project: http://git-wip-us.apache.org/repos/asf/chukwa/repo Commit: http://git-wip-us.apache.org/repos/asf/chukwa/commit/0e518f8d Tree: http://git-wip-us.apache.org/repos/asf/chukwa/tree/0e518f8d Diff: http://git-wip-us.apache.org/repos/asf/chukwa/diff/0e518f8d Branch: refs/heads/master Commit: 0e518f8d71a3bba3534ab068e6703386e32f4c23 Parents: 5e66273 Author: Eric Yang <[email protected]> Authored: Sun Nov 22 17:19:42 2015 -0800 Committer: Eric Yang <[email protected]> Committed: Sun Nov 22 17:19:42 2015 -0800 ---------------------------------------------------------------------- CHANGES.txt | 2 + pom.xml | 13 ++- src/main/web/hicc/css/welcome.css | 39 ++++++++ src/main/web/hicc/welcome.html | 147 +++++++++++++++++++++++++++- src/site/apt/Quick_Start_Guide.apt.vm | 2 +- src/site/apt/agent.apt | 25 +++-- src/site/apt/datamodel.apt | 54 +++++++++++ src/site/apt/design.apt | 50 +--------- src/site/apt/hicc.apt | 149 ----------------------------- src/site/apt/releasenotes.apt | 52 ---------- src/site/apt/releasenotes.apt.vm | 45 +++++++++ src/site/site.xml | 3 +- 12 files changed, 320 insertions(+), 261 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/chukwa/blob/0e518f8d/CHANGES.txt ---------------------------------------------------------------------- diff --git a/CHANGES.txt b/CHANGES.txt index 718d869..cab670f 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -28,6 +28,8 @@ Trunk (unreleased changes) IMPROVEMENTS + CHUKWA-786. Update documentation to reflect 0.7 release. (Eric Yang) + CHUKWA-773. Update maven surefire version. (Anna Wang via Eric Yang) CHUKWA-769. Updated Chukwa to current HBase API. (Eric Yang) http://git-wip-us.apache.org/repos/asf/chukwa/blob/0e518f8d/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 8046179..d078072 100644 --- a/pom.xml +++ b/pom.xml @@ -49,6 +49,7 @@ <zookeeperVersion>3.4.5</zookeeperVersion> <hbaseVersion>1.0.0</hbaseVersion> <hadoopVersion>2.6.0</hadoopVersion> + <solrVersion>4.7.2</solrVersion> <!-- <JAVA_HOME>${java.home}</JAVA_HOME> --> </properties> @@ -293,7 +294,7 @@ <dependency> <groupId>org.apache.solr</groupId> <artifactId>solr-test-framework</artifactId> - <version>4.7.2</version> + <version>${solrVersion}</version> <type>jar</type> <scope>test</scope> <exclusions> @@ -739,6 +740,16 @@ <copy todir="${project.build.directory}/site"> <fileset dir="src/site/resources/css" includes="*.css" /> </copy> + <copy todir="${project.build.directory}/site/css"> + <fileset dir="src/main/web/hicc/css" includes="*" /> + </copy> + <copy todir="${project.build.directory}/site/js"> + <fileset dir="src/main/web/hicc/js" includes="*" /> + </copy> + <copy todir="${project.build.directory}/site/fonts"> + <fileset dir="src/main/web/hicc/fonts" includes="*" /> + </copy> + <copy file="src/main/web/hicc/welcome.html" tofile="${project.build.directory}/site/hicc.html" /> </tasks> </configuration> </execution> http://git-wip-us.apache.org/repos/asf/chukwa/blob/0e518f8d/src/main/web/hicc/css/welcome.css ---------------------------------------------------------------------- diff --git a/src/main/web/hicc/css/welcome.css b/src/main/web/hicc/css/welcome.css new file mode 100644 index 0000000..b50f683 --- /dev/null +++ b/src/main/web/hicc/css/welcome.css @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +.content section { + background-color: #eeeeee; + margin-bottom:20px; + height:750px; +} + +#side-nav { + left: 95%; + height: 300px; + position: absolute; + top: 40%; + z-index: 10; +} + +.nav-list > .active { + background-color: #0088cc; +} + +.nav-list > .active > a { + color: #ffffff; +} http://git-wip-us.apache.org/repos/asf/chukwa/blob/0e518f8d/src/main/web/hicc/welcome.html ---------------------------------------------------------------------- diff --git a/src/main/web/hicc/welcome.html b/src/main/web/hicc/welcome.html index 5b44d94..50254b9 100644 --- a/src/main/web/hicc/welcome.html +++ b/src/main/web/hicc/welcome.html @@ -14,4 +14,149 @@ See the License for the specific language governing permissions and limitations under the License. --> -Welcome to Hadoop Infrastructure Care Center. +<!DOCTYPE html> +<html lang="en" class="no-js"> + <head> + <link href="css/bootstrap-theme.min.css" type="text/css" rel="stylesheet" /> + <link href="css/bootstrap.min.css" type="text/css" rel="stylesheet" /> + <link href="css/welcome.css" type="text/css" rel="stylesheet" /> + <script src="js/jquery.js" type="text/javascript"></script> + <script src="js/bootstrap.min.js" type="text/javascript"></script> + </head> + <body data-spy="scroll" data-target="#side-nav"> + +<div class="container"> + <div class="row"> + <div id="side-nav" class="span3"> + <ul class="nav nav-list affix"> + <li><a href="#one"><span class="glyphicon glyphicon-unchecked"></span></a></li> + <li><a href="#two"><span class="glyphicon glyphicon-unchecked"></span></a></li> + <li><a href="#three"><span class="glyphicon glyphicon-unchecked"></span></a></li> + <li><a href="#four"><span class="glyphicon glyphicon-unchecked"></span></a></li> + <li><a href="#five"><span class="glyphicon glyphicon-unchecked"></span></a></li> + </ul> + </div> + <div class="span9 content"> + <section id="one"> + <div class="jumbotron text-center"> + <h1> + Manage services and users and monitor your cluster with the Hadoop Infrastructure Care Center dashboard + </h1> + <br><br> + <p> + This starter kit gets you started using the HICC dashboard to manage and monitor your clusters. + By the end of this starter kit, you'll be familiar with the steps to work with home page, + monitor activities, set up views, and define widgets.</p> + </div> + </section> + <section id="two"> + <div class="jumbotron"> + <h1>Home</h1> + <br><br> + <p> + Home is a user customizable home page. Use top right <span class="glyphicon glyphicon-menu-hamburger"></span> + menu button to bring up navigation. Search function assists to find widgets in HICC Widget catalog. + When hover over a widget, user have 4 options to interact with the widgets. <span class="glyphicon glyphicon-remove"></span> + on the menu bar will close the Widget. <span class="glyphicon glyphicon-new-window"></span> will + zoom the widget to full page view. Widget bar enables dragging placement of widget in the dashboard. + By dragging the bottom right corner of the widget, the widget can be resized. + </p> + <p> + At top menu bar, H D M Y are representation buttons of Time range. Each button presents a different time scale. + The available time scale are: <b>H</b>our, <b>D</b>ay, <b>M</b>onth, and <b>Y</b>ear. By pressing the button, + the dashboard displays information corresponding to the selected time scale. + </p> + <p> + The top right <span class="glyphicon glyphicon-off"></span> button is the logout button. + </p> + </div> + </section> + <section id="three"> + <div class="jumbotron"> + <h1>User Activities</h1> + <br><br> + <p> + User Activities dashboard displays Administrator defined dashboard for visualizing the user + related information. + </p> + <p> + User generated activities, such as running, deploying and administering an + application. Log search helps to provide ability to search for key words that help user + to correlate events together. Other activities such as metering of application usage + also help users to gain insights about their usage pattern. From usage pattern, application + administrator can improve capacity plan for various type of applications. + </p> + </div> + </section> + <section id="four"> + <div class="jumbotron"> + <h1>System Activities</h1> + <br><br> + <p> + System Activities dashboard displays Administrator defined dashboard for visualizing the overall health + of the infrastructure. + </p> + <p> + The default system dashboard displays cluster, system, and services health. This dashboard + helps to monitor the cluster health, and amount of resource usage. These information can + help system administrators to improve infrastructure quality. + </p> + </div> + </section> + <section id="five"> + <div class="jumbotron"> + <h1>Graph Explorer</h1> + <br><br> + <p> + Graph Explorer is a widget composer for data scientist to correlate metrics data and generate chart and graphs. + </p> + <h2>Metrics Group</h2> + <p> + Metrics Group is a kind of data collected by Chukwa Agent. When a data stream is collected from the source, data type + is predefined to ensure Chukwa can use the correct parser to process the incoming data type. When user wants to compose + a new widget, metrics group is used to narrow down which group the information belongs to. + </p> + <h2>Metrics</h2> + <p> + Metrics is a list of possible metrics from a metrics group. User can select multiple metrics to visualize correlation + between metrics. For example, for correlation of total disk uage, user can select SystemMetrics.disk.Total and + SystemMetrics.disk.Used to visualize the disk usage of a given node. + </p> + <h2>Sources</h2> + <p> + Source refers to the origin of the data source. It is a high level concept to representing the source of the data + is originated from a node, a service, a process, or a device. This enables user to selectively filter the source of the metrics + and visualize individual source or all sources. + </p> + <h2>Title</h2> + <p> + Title defines searchable name when widget is published to Widget catalog. + </p> + <h2>Time Series</h2> + <p> + Visualize time series metrics in a 2 dimension xy chart. User can customize the visualization to plot line, bar, area, and + scatter chart. + </p> + <h2>Tile</h2> + <p> + Display a banner of the last known value. User can customize tile to display either a string, or numerical value. + </p> + <h2>Pie Chart</h2> + <p> + Pie chart visualization is designed to visualize pie chart base on pre-defined metrics type. User can select multiple + metrics and the averaged value of the selected time range is used to compute the values of each metrics on the pie chart. + </p> + <h2>Ring Chart</h2> + <p> + Ring chart visualization is suitable for visualize guage value to ensure specific metrics does not exceed its maximum + value. A single metrics of 100% percent or a pair of metrics that representing current / maximum values can be specified + to visualize the ring chart. + </p> + </div> + </section> + </div> + </div> +</div> + + </body> +</html> http://git-wip-us.apache.org/repos/asf/chukwa/blob/0e518f8d/src/site/apt/Quick_Start_Guide.apt.vm ---------------------------------------------------------------------- diff --git a/src/site/apt/Quick_Start_Guide.apt.vm b/src/site/apt/Quick_Start_Guide.apt.vm index 0019d1f..6248eb6 100644 --- a/src/site/apt/Quick_Start_Guide.apt.vm +++ b/src/site/apt/Quick_Start_Guide.apt.vm @@ -24,7 +24,7 @@ Pre-requisites Chukwa should work on any POSIX platform, but GNU/Linux is the only production platform that has been tested extensively. Chukwa has also been used successfully on Mac OS X, which several members of the Chukwa team use for development. - The only absolute software requirements are Java 1.6 or better, ZooKeeper {{${zookeeperVersion}}}, HBase {{${hbaseVersion}}} and Hadoop {{${hadoopVersion}}}. + Software requirements are Java 1.6 or better, ZooKeeper {{${zookeeperVersion}}}, HBase {{${hbaseVersion}}} and Hadoop {{${hadoopVersion}}}. The Chukwa cluster management scripts rely on ssh; these scripts, however, are not required if you have some alternate mechanism for starting and stopping daemons. http://git-wip-us.apache.org/repos/asf/chukwa/blob/0e518f8d/src/site/apt/agent.apt ---------------------------------------------------------------------- diff --git a/src/site/apt/agent.apt b/src/site/apt/agent.apt index 79cbf41..e002f25 100644 --- a/src/site/apt/agent.apt +++ b/src/site/apt/agent.apt @@ -82,16 +82,27 @@ be added automatically. Adaptor parameters aren't required by the Chukwa agent, but each class of adaptor may itself specify both mandatory and optional parameters. See below. -Command-line options +Configuration options - Normally, agents are configured via the file <conf/chukwa-agent-conf.xml.> -However, there are a few command-line options that are sometimes useful in -troubleshooting. If you specify "local" as an option, then the agent will print -chunks to standard out, rather than to pipeline writers. -This option is intended for testing and debugging, not for production use. + Chukwa agents are configured via the file <conf/chukwa-agent-conf.xml.> +Chukwa control port runs on port 9093 by default. --- -bin/chukwa agent local + <property> + <name>chukwaAgent.control.port</name> + <value>9093</value> + <description>The socket port number the agent's control interface can be contacted at.</description> + </property> +--- + + Chukwa agent working directory: + +--- + <property> + <name>chukwaAgent.checkpoint.dir</name> + <value>${CHUKWA_LOG_DIR}/</value> + <description>the location to put the agent's checkpoint file(s)</description> + </property> --- Adaptors http://git-wip-us.apache.org/repos/asf/chukwa/blob/0e518f8d/src/site/apt/datamodel.apt ---------------------------------------------------------------------- diff --git a/src/site/apt/datamodel.apt b/src/site/apt/datamodel.apt new file mode 100644 index 0000000..1aae1f5 --- /dev/null +++ b/src/site/apt/datamodel.apt @@ -0,0 +1,54 @@ +~~ Licensed to the Apache Software Foundation (ASF) under one or more +~~ contributor license agreements. See the NOTICE file distributed with +~~ this work for additional information regarding copyright ownership. +~~ The ASF licenses this file to You under the Apache License, Version 2.0 +~~ (the "License"); you may not use this file except in compliance with +~~ the License. You may obtain a copy of the License at +~~ +~~ http://www.apache.org/licenses/LICENSE-2.0 +~~ +~~ Unless required by applicable law or agreed to in writing, software +~~ distributed under the License is distributed on an "AS IS" BASIS, +~~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +~~ See the License for the specific language governing permissions and +~~ limitations under the License. +~~ + +Data Model + + Chukwa Adaptors emit data in <Chunks>. A Chunk is a sequence of bytes, +with some metadata. Several of these are set automatically by the Agent or +Adaptors. Two of them require user intervention: <cluster name> and +<datatype>. Cluster name is specified in <conf/chukwa-agent-conf.xml>, +and is global to each Agent process. Datatype describes the expected format +of the data collected by an Adaptor instance, and it is specified when that +instance is started. + + The following table lists the Chunk metadata fields: + +*-------------*-------------------------------------+----------------------------------------------------------: +| Field | Meaning | Source | +*-------------*-------------------------------------+----------------------------------------------------------: +| Source | Hostname where Chunk was generated | Automatic | +*-------------*-------------------------------------+----------------------------------------------------------: +| Cluster | Cluster host is associated with | Specified by user in agent config | +*-------------*-------------------------------------+----------------------------------------------------------: +| Datatype | Format of output | Specified by user when Adaptor started | +*-------------*-------------------------------------+----------------------------------------------------------: +| Sequence ID | Offset of Chunk in stream | Automatic, initial offset specified when Adaptor started | +*-------------*-------------------------------------+----------------------------------------------------------: +| Name | Name of data source | Automatic, chosen by Adaptor | +*-------------*-------------------------------------+----------------------------------------------------------: + + Conceptually, each Adaptor emits a semi-infinite stream of bytes, numbered +starting from zero. The sequence ID specifies how many bytes each Adaptor has +sent, including the current chunk. So if an adaptor emits a chunk containing +the first 100 bytes from a file, the sequenceID of that Chunk will be 100. +And the second hundred bytes will have sequence ID 200. This may seem a +little peculiar, but it's actually the same way that TCP sequence numbers work. + + Adaptors need to take sequence ID as a parameter so that they can resume +correctly after a crash, and not send redundant data. When starting adaptors, +it's usually save to specify 0 as an ID, but it's sometimes useful to specify +something else. For instance, it lets you do things like only tail the second +half of a file. http://git-wip-us.apache.org/repos/asf/chukwa/blob/0e518f8d/src/site/apt/design.apt ---------------------------------------------------------------------- diff --git a/src/site/apt/design.apt b/src/site/apt/design.apt index c5ac4ce..4122fe6 100644 --- a/src/site/apt/design.apt +++ b/src/site/apt/design.apt @@ -62,45 +62,6 @@ must be monitored. We might want to increase our collection rate if we detect anomalies. And of course, it makes no sense to collect Hadoop metrics on an NFS server. -Data Model - - Chukwa Adaptors emit data in <Chunks>. A Chunk is a sequence of bytes, -with some metadata. Several of these are set automatically by the Agent or -Adaptors. Two of them require user intervention: <cluster name> and -<datatype>. Cluster name is specified in <conf/chukwa-agent-conf.xml>, -and is global to each Agent process. Datatype describes the expected format -of the data collected by an Adaptor instance, and it is specified when that -instance is started. - -The following table lists the Chunk metadata fields. - -*-------------*-------------------------------------+----------------------------------------------------------: -| Field | Meaning | Source | -*-------------*-------------------------------------+----------------------------------------------------------: -| Source | Hostname where Chunk was generated | Automatic | -*-------------*-------------------------------------+----------------------------------------------------------: -| Cluster | Cluster host is associated with | Specified by user in agent config | -*-------------*-------------------------------------+----------------------------------------------------------: -| Datatype | Format of output | Specified by user when Adaptor started | -*-------------*-------------------------------------+----------------------------------------------------------: -| Sequence ID | Offset of Chunk in stream | Automatic, initial offset specified when Adaptor started | -*-------------*-------------------------------------+----------------------------------------------------------: -| Name | Name of data source | Automatic, chosen by Adaptor | -*-------------*-------------------------------------+----------------------------------------------------------: - - Conceptually, each Adaptor emits a semi-infinite stream of bytes, numbered -starting from zero. The sequence ID specifies how many bytes each Adaptor has -sent, including the current chunk. So if an adaptor emits a chunk containing -the first 100 bytes from a file, the sequenceID of that Chunk will be 100. -And the second hundred bytes will have sequence ID 200. This may seem a -little peculiar, but it's actually the same way that TCP sequence numbers work. - - Adaptors need to take sequence ID as a parameter so that they can resume -correctly after a crash, and not send redundant data. When starting adaptors, -it's usually save to specify 0 as an ID, but it's sometimes useful to specify -something else. For instance, it lets you do things like only tail the second -half of a file. - ETL Processes Chukwa Agents can write data directly to HBase or sequence files. @@ -146,16 +107,7 @@ which in turn is populated by collector or data analytic scripts that runs on the collected data, after Demux. The {{{./admin.html}Administration guide}} has details on setting up HICC. -Collectors (Deprecated) - - The origin design of collector is to reduce the number of TCP connections -for collecting data from various source, provide high availability and wire -compatibility across versions. Data transfer reliability has been improved -in HDFS client and HBase client. The original problem that Chukwa Collector -tried to solve is no longer high priority in Chukwa data collection framework -because both HDFS and HBase are in better position in solving data transport -and replication problem. Hence, datanode and HBase Region servers are -replacement for Chukwa collectors. +HBase Integration Chukwa has adopted to use HBase to ensure data arrival in milli-seconds and also make data available to down steam application at the same time. This http://git-wip-us.apache.org/repos/asf/chukwa/blob/0e518f8d/src/site/apt/hicc.apt ---------------------------------------------------------------------- diff --git a/src/site/apt/hicc.apt b/src/site/apt/hicc.apt deleted file mode 100644 index 9c3b3ee..0000000 --- a/src/site/apt/hicc.apt +++ /dev/null @@ -1,149 +0,0 @@ -~~ Licensed to the Apache Software Foundation (ASF) under one or more -~~ contributor license agreements. See the NOTICE file distributed with -~~ this work for additional information regarding copyright ownership. -~~ The ASF licenses this file to You under the Apache License, Version 2.0 -~~ (the "License"); you may not use this file except in compliance with -~~ the License. You may obtain a copy of the License at -~~ -~~ http://www.apache.org/licenses/LICENSE-2.0 -~~ -~~ Unless required by applicable law or agreed to in writing, software -~~ distributed under the License is distributed on an "AS IS" BASIS, -~~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -~~ See the License for the specific language governing permissions and -~~ limitations under the License. -~~ - -HICC Operation Manual - - HICC stands for Hadoop Infrastructure Care Center. It is the central dashboard -for visualize and monitoring of metrics collected by Chukwa. - -How HICC is organized - - To get started with HICC, you should get to know the menus and tools of the -dashboard, and learn how widgets and information are organized. - -* Dashboard - - When you login to HICC, the dashboard appears. You'll see a menu bar along -the top of the browser window, and widgets in rows and columns. In between -the menu bar and the widgets is the tabs. In the upper right corner of dashboard -are the icons for HICC global options. - -[./images/hicc-dashboard.png] - -* How widgets are organized - - HICC organizes widgets in Widgets Catalog where information are organized in -a tree structure. Some of the categories are described below. - -[./images/hicc-widget.png] - - * <<Developer>> A list of utility widgets for exploring metrics stored on HBase - and debugging tools to help developer to develope new widgets. - - * <<Global>> A list of utility widgets for user to control time, cluster, or - host selection. - - * <<Hadoop>> Hadoop related metrics and job summary are organized in this - category. - - * <<System Metrics>> System related metrics such as cpu, disk, network or - memory utilization are organized in this category. - -* Navigating HICC - - The items stored on HICC -- including applications, utilities and widgets -- -are organized in a hierarchy of widget category that you can browse by accessing -menu bar "Options". The dashboard has columns and rows and you can rearrange -the layout of the widgets by drag and drop. Rearranging widgets on the dashboard -is only saved when "Save Dashboard" is click. - -Menu, tabs and dashboards - -* Menu basics - - Menus contain options that perform actions. For example, to create an extra -column on the dashboard, you choose the add column command from the Options -menu (Options > Add Column). - -** Opening menus: - - * To open a menu in the menu bar across the top of the dashboard, hover over -the menu item. - - * Many menu item is also a toggle. For example, choose Dashboard Builder will -open the view for a list of available dashboards. Choose Dashboard Builder a -second time will close Dashboard list view. - -** Choosing menu options: - - Click the option you want to use. - - * If the option has an ellipsis (...) next to it, a dialog opens where you can -make additional choices or provide any necessary information. - - * If an option is dimmed, it is not available. For example, you can not edit -name of the dashboard, if you are not the owner of the dashboard. - -* Dashboard basics - - Each dashboard can have multiple tabs and each tab is divided into row and columns. -You can add more widgets by selecting Options > choose a widget > click Add. -To remove a widgets by clicking on Close button on the top right hand corner of -the widget. - -* Tab basics - - Tab provides a way to organize related information together. As you create widgets, -manange cluster and do other work on HICC, you can create new tabs to keep your -information organized. - - * To create a new tab: - - [[1]] Select "Add New Tab" from menu bar (Options > Add New Tab). - - [[2]] Enter a new name for the tab and press Return. - - [] - - * To remove a tab: - - [[1]] Select the tab that you wish to delete. - - [[2]] Select delete current tab from menu bar (Options > Delete Current Tab). - - [] - - * To rename a tab: - - [[1]] Double click on the tab to change tab to edit mode. - - [[2]] Enter a new name for the tab and press Return. - -User accounts - - You should set up an account for each person who uses HICC on a regular basis. - -* Create a new user account - - [[1]] Edit CHUKWA_CONF_DIR/auth.conf - - [[2]] Authentication and authorization information is stored in a properties file. -Eacho line in the file contains a username, a password and 0 or more role assignment. -The format is: - ---- -username: password[,rolename ...] ---- - -* What is an administrator account? - - Administrator account has privileges to modify all dashboard layout and change -read/write permission of the dashboards. - -* What is a user account? - - User account has privileges to modify user's own dashboard layout and change -read/write permission of his/her own dashboard. http://git-wip-us.apache.org/repos/asf/chukwa/blob/0e518f8d/src/site/apt/releasenotes.apt ---------------------------------------------------------------------- diff --git a/src/site/apt/releasenotes.apt b/src/site/apt/releasenotes.apt deleted file mode 100644 index 36f80c5..0000000 --- a/src/site/apt/releasenotes.apt +++ /dev/null @@ -1,52 +0,0 @@ -~~ Licensed to the Apache Software Foundation (ASF) under one or more -~~ contributor license agreements. See the NOTICE file distributed with -~~ this work for additional information regarding copyright ownership. -~~ The ASF licenses this file to You under the Apache License, Version 2.0 -~~ (the "License"); you may not use this file except in compliance with -~~ the License. You may obtain a copy of the License at -~~ -~~ http://www.apache.org/licenses/LICENSE-2.0 -~~ -~~ Unless required by applicable law or agreed to in writing, software -~~ distributed under the License is distributed on an "AS IS" BASIS, -~~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -~~ See the License for the specific language governing permissions and -~~ limitations under the License. -~~ -Release Notes - Hadoop Chukwa - Version 0.6 - -Overall Status - - This is the fourth public release of Chukwa, a log analysis framework on top of Hadoop -and HBase. Chukwa has been tested at scale and used in some production settings, and -is reasonably robust and well behaved. For instructions on setting up Chukwa, see the -administration guide and the rest of the Chukwa documentation. - -Important Changes Since Last Release - - * Chukwa can store data on HBase for improved low latency read performance. - - * Chukwa collector has been deprecated. HBase and HDFS can support better reliability -for data injection. The original purpose of collector is superseded by both services. - - * There have been a number of bug fixes and code cleanups since the last release; - check the changelog and JIRA for details. - -Requirements - - Chukwa relies on Java 1.6, and requires maven 3.0.3 to build. -The back-end processing requires Hadoop 1.2.1+, HBase 0.96+, and Pig 0.9.1+. - -Known Limitations - - * HICC defaults to assuming data is UTC; if your machines run on local time, - HICC graphs will not display properly until you change the HICC timezone. - You can do this by clicking the small "gear" icon on the time selection tool. - - * As mentioned in the administration guide, the pig aggregation script requires - external scheduling in cronjob or Jenkins. - - * Salsa Finite State Machine has not been ported forward to store data on HBase. - - * There is currently no down sampling script for data stored on HBase. Long term - trending visualization might not work for large scale data. http://git-wip-us.apache.org/repos/asf/chukwa/blob/0e518f8d/src/site/apt/releasenotes.apt.vm ---------------------------------------------------------------------- diff --git a/src/site/apt/releasenotes.apt.vm b/src/site/apt/releasenotes.apt.vm new file mode 100644 index 0000000..590867a --- /dev/null +++ b/src/site/apt/releasenotes.apt.vm @@ -0,0 +1,45 @@ +~~ Licensed to the Apache Software Foundation (ASF) under one or more +~~ contributor license agreements. See the NOTICE file distributed with +~~ this work for additional information regarding copyright ownership. +~~ The ASF licenses this file to You under the Apache License, Version 2.0 +~~ (the "License"); you may not use this file except in compliance with +~~ the License. You may obtain a copy of the License at +~~ +~~ http://www.apache.org/licenses/LICENSE-2.0 +~~ +~~ Unless required by applicable law or agreed to in writing, software +~~ distributed under the License is distributed on an "AS IS" BASIS, +~~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +~~ See the License for the specific language governing permissions and +~~ limitations under the License. +~~ +Release Notes - Hadoop Chukwa - Version ${VERSION} + +Overall Status + + This is the fifth public release of Chukwa, a log analysis framework on top of Hadoop +and HBase. Chukwa has been tested at scale and used in some production settings, and +is reasonably robust and well behaved. For instructions on setting up Chukwa, see the +administration guide and the rest of the Chukwa documentation. + +Important Changes Since Last Release + + * New dashboard design. + + * New Chukwa Parquet file format. + + * New HBase schema for improved low latency read performance. + + * New Solr log indexing support. + + * There have been a number of bug fixes and code cleanups since the last release; + check the changelog and JIRA for details. + +Requirements + + Chukwa relies on Java 1.6, and requires maven 3.0.3 to build. +The back-end processing requires Hadoop ${hadoopVersion}, HBase ${hbaseVersion}+, and Solr ${solrVersion}+. + +Known Limitations + + * There is currently no HDFS file compaction and retention support. http://git-wip-us.apache.org/repos/asf/chukwa/blob/0e518f8d/src/site/site.xml ---------------------------------------------------------------------- diff --git a/src/site/site.xml b/src/site/site.xml index b369959..a5b29a4 100644 --- a/src/site/site.xml +++ b/src/site/site.xml @@ -52,7 +52,8 @@ <item name="Javadocs" href="apidocs/index.html"/> </item> <item name="Architecture" href="design.html"> - <item name="HDFS Layout" href="dataflow.html"/> + <item name="Data Model" href="datamodel.html"/> + <!--<item name="HDFS Layout" href="dataflow.html"/>--> </item> <item name="Wiki" href="http://wiki.apache.org/hadoop/Chukwa/"/> <item name="FAQ" href="http://wiki.apache.org/hadoop/Chukwa/FAQ"/>
