Repository: carbondata-site Updated Branches: refs/heads/asf-site 61436c8e4 -> 44fe8a42d
update md documents Project: http://git-wip-us.apache.org/repos/asf/carbondata-site/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata-site/commit/44fe8a42 Tree: http://git-wip-us.apache.org/repos/asf/carbondata-site/tree/44fe8a42 Diff: http://git-wip-us.apache.org/repos/asf/carbondata-site/diff/44fe8a42 Branch: refs/heads/asf-site Commit: 44fe8a42d2101a5ec0ae6924ac5d712b72debc94 Parents: 61436c8 Author: chenliang613 <[email protected]> Authored: Sat Jun 2 12:01:37 2018 +0800 Committer: chenliang613 <[email protected]> Committed: Sat Jun 2 12:01:37 2018 +0800 ---------------------------------------------------------------------- content/WEB-INF/classes/MdFileHandler.class | Bin 6144 -> 6144 bytes content/WEB-INF/classes/html/header.html | 3 ++ content/data-management-on-carbondata.html | 4 +- content/index.html | 7 +++ content/sdk-guide.html | 42 ++++++++++++++---- content/timeseries-datamap-guide.html | 8 ++-- .../webapp/data-management-on-carbondata.html | 4 +- src/main/webapp/sdk-guide.html | 42 ++++++++++++++---- src/main/webapp/timeseries-datamap-guide.html | 8 ++-- src/site/markdown/sdk-guide.md | 43 +++++++++++++++---- 10 files changed, 125 insertions(+), 36 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/content/WEB-INF/classes/MdFileHandler.class ---------------------------------------------------------------------- diff --git a/content/WEB-INF/classes/MdFileHandler.class b/content/WEB-INF/classes/MdFileHandler.class index f39c098..58088b3 100644 Binary files a/content/WEB-INF/classes/MdFileHandler.class and b/content/WEB-INF/classes/MdFileHandler.class differ http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/content/WEB-INF/classes/html/header.html ---------------------------------------------------------------------- diff --git a/content/WEB-INF/classes/html/header.html b/content/WEB-INF/classes/html/header.html index d2ebd58..470de01 100644 --- a/content/WEB-INF/classes/html/header.html +++ b/content/WEB-INF/classes/html/header.html @@ -51,6 +51,9 @@ aria-expanded="false"> Download <span class="caret"></span></a> <ul class="dropdown-menu"> <li> + <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"; + target="_blank">Apache CarbonData 1.4.0</a></li> + <li> <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"; target="_blank">Apache CarbonData 1.3.1</a></li> <li> http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/content/data-management-on-carbondata.html ---------------------------------------------------------------------- diff --git a/content/data-management-on-carbondata.html b/content/data-management-on-carbondata.html index a9c7926..c81b02c 100644 --- a/content/data-management-on-carbondata.html +++ b/content/data-management-on-carbondata.html @@ -195,11 +195,11 @@ <a id="create-table" class="anchor" href="#create-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CREATE TABLE</h2> <p>This command can be used to create a CarbonData table by specifying the list of fields along with the table properties. You can also specify the location where the table needs to be stored.</p> <pre><code>CREATE TABLE [IF NOT EXISTS] [db_name.]table_name[(col_name data_type , ...)] -STORED BY 'carbondata' +STORED AS carbondata [TBLPROPERTIES (property_name=property_value, ...)] [LOCATION 'path'] </code></pre> -<p><strong>NOTE:</strong> CarbonData also supports "STORED AS carbondata". Find example code at <a href="https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonSessionExample.scala"; target=_blank>CarbonSessionExample</a> in the CarbonData repo.</p> +<p><strong>NOTE:</strong> CarbonData also supports "STORED AS carbondata" and "USING carbondata". Find example code at <a href="https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonSessionExample.scala"; target=_blank>CarbonSessionExample</a> in the CarbonData repo.</p> <h3> <a id="usage-guidelines" class="anchor" href="#usage-guidelines" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h3> <p>Following are the guidelines for TBLPROPERTIES, CarbonData's additional table options can be set via carbon.properties.</p> http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/content/index.html ---------------------------------------------------------------------- diff --git a/content/index.html b/content/index.html index c201fb2..c1ea996 100644 --- a/content/index.html +++ b/content/index.html @@ -328,6 +328,13 @@ </h4> <div class="linkblock"> <div class="block-row"> + <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"; + target="_blank">Apache CarbonData 1.4.0</a> + <span class="release-date">Mar 2018</span> + <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+1.4.0+Release"; + class="whatsnew" target="_blank">what's new</a> + </div> + <div class="block-row"> <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"; target="_blank">Apache CarbonData 1.3.1</a> <span class="release-date">Mar 2018</span> http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/content/sdk-guide.html ---------------------------------------------------------------------- diff --git a/content/sdk-guide.html b/content/sdk-guide.html index 08044d1..7128c6c 100644 --- a/content/sdk-guide.html +++ b/content/sdk-guide.html @@ -340,6 +340,14 @@ These SDK writer output contains just a carbondata and carbonindex files. No met </tbody> </table> <h2> +<a id="run-sql-on-files-directly" class="anchor" href="#run-sql-on-files-directly" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Run SQL on files directly</h2> +<p>Instead of creating table and query it, you can also query that file directly with SQL.</p> +<h3> +<a id="example" class="anchor" href="#example" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Example</h3> +<pre><code>SELECT * FROM carbonfile.`$Path` +</code></pre> +<p>Find example code at <a href="https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/DirectSQLExample.scala"; target=_blank>DirectSQLExample</a> in the CarbonData repo.</p> +<h2> <a id="api-list" class="anchor" href="#api-list" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>API List</h2> <h3> <a id="class-orgapachecarbondatasdkfilecarbonwriterbuilder" class="anchor" href="#class-orgapachecarbondatasdkfilecarbonwriterbuilder" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Class org.apache.carbondata.sdk.file.CarbonWriterBuilder</h3> @@ -578,17 +586,22 @@ External client can make use of this reader to read CarbonData files without Car String path = "./testWriteFiles"; CarbonReader reader = CarbonReader .builder(path, "_temp") - .projection(new String[]{"name", "age"}) + .projection(new String[]{"stringField", "shortField", "intField", "longField", + "doubleField", "boolField", "dateField", "timeField", "decimalField"}) .build(); // 2. Read data + long day = 24L * 3600 * 1000; int i = 0; while (reader.hasNext()) { - Object[] row = (Object[]) reader.readNextRow(); - System.out.println(row[0] + "\t" + row[1]); - i++; + Object[] row = (Object[]) reader.readNextRow(); + System.out.println(String.format("%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t", + i, row[0], row[1], row[2], row[3], row[4], row[5], + new Date((day * ((int) row[6]))), new Timestamp((long) row[7] / 1000), row[8] + )); + i++; } - + // 3. Close this reader reader.close(); </code></pre> @@ -597,12 +610,25 @@ External client can make use of this reader to read CarbonData files without Car <a id="api-list-1" class="anchor" href="#api-list-1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>API List</h2> <h3> <a id="class-orgapachecarbondatasdkfilecarbonreader" class="anchor" href="#class-orgapachecarbondatasdkfilecarbonreader" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Class org.apache.carbondata.sdk.file.CarbonReader</h3> -<pre><code> /** - * Return a new CarbonReaderBuilder instance - */ +<pre><code> /** + * Return a new {@link CarbonReaderBuilder} instance + * + * @param tablePath table store path + * @param tableName table name + * @return CarbonReaderBuilder object + */ public static CarbonReaderBuilder builder(String tablePath, String tableName); </code></pre> <pre><code> /** + * Return a new CarbonReaderBuilder instance + * Default value of table name is table + tablePath + time + * + * @param tablePath table path + * @return CarbonReaderBuilder object + */ + public static CarbonReaderBuilder builder(String tablePath); +</code></pre> +<pre><code> /** * Return true if has next row */ public boolean hasNext(); http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/content/timeseries-datamap-guide.html ---------------------------------------------------------------------- diff --git a/content/timeseries-datamap-guide.html b/content/timeseries-datamap-guide.html index b40150c..2400ce0 100644 --- a/content/timeseries-datamap-guide.html +++ b/content/timeseries-datamap-guide.html @@ -179,14 +179,14 @@ <div><h1> <a id="carbondata-timeseries-datamap" class="anchor" href="#carbondata-timeseries-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CarbonData Timeseries DataMap</h1> <ul> -<li><a href="#timeseries-datamap-intoduction-(alpha-feature-in-1.3.0)">Timeseries DataMap</a></li> +<li><a href="#timeseries-datamap-intoduction">Timeseries DataMap Introduction</a></li> <li><a href="#compacting-pre-aggregate-tables">Compaction</a></li> <li><a href="#data-management-with-pre-aggregate-tables">Data Management</a></li> </ul> <h2> -<a id="timeseries-datamap-intoduction-alpha-feature-in-130" class="anchor" href="#timeseries-datamap-intoduction-alpha-feature-in-130" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Timeseries DataMap Intoduction (Alpha feature in 1.3.0)</h2> -<p>Timeseries DataMap a pre-aggregate table implementation based on 'preaggregate' DataMap. -Difference is that Timerseries DataMap has built-in understanding of time hierarchy and +<a id="timeseries-datamap-introduction-alpha-feature-in-130" class="anchor" href="#timeseries-datamap-introduction-alpha-feature-in-130" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Timeseries DataMap Introduction (Alpha feature in 1.3.0)</h2> +<p>Timeseries DataMap a pre-aggregate table implementation based on 'pre-aggregate' DataMap. +Difference is that Timeseries DataMap has built-in understanding of time hierarchy and levels: year, month, day, hour, minute, so that it supports automatic roll-up in time dimension for query.</p> <p>The data loading, querying, compaction command and its behavior is the same as preaggregate DataMap. http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/src/main/webapp/data-management-on-carbondata.html ---------------------------------------------------------------------- diff --git a/src/main/webapp/data-management-on-carbondata.html b/src/main/webapp/data-management-on-carbondata.html index a9c7926..c81b02c 100644 --- a/src/main/webapp/data-management-on-carbondata.html +++ b/src/main/webapp/data-management-on-carbondata.html @@ -195,11 +195,11 @@ <a id="create-table" class="anchor" href="#create-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CREATE TABLE</h2> <p>This command can be used to create a CarbonData table by specifying the list of fields along with the table properties. You can also specify the location where the table needs to be stored.</p> <pre><code>CREATE TABLE [IF NOT EXISTS] [db_name.]table_name[(col_name data_type , ...)] -STORED BY 'carbondata' +STORED AS carbondata [TBLPROPERTIES (property_name=property_value, ...)] [LOCATION 'path'] </code></pre> -<p><strong>NOTE:</strong> CarbonData also supports "STORED AS carbondata". Find example code at <a href="https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonSessionExample.scala"; target=_blank>CarbonSessionExample</a> in the CarbonData repo.</p> +<p><strong>NOTE:</strong> CarbonData also supports "STORED AS carbondata" and "USING carbondata". Find example code at <a href="https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonSessionExample.scala"; target=_blank>CarbonSessionExample</a> in the CarbonData repo.</p> <h3> <a id="usage-guidelines" class="anchor" href="#usage-guidelines" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h3> <p>Following are the guidelines for TBLPROPERTIES, CarbonData's additional table options can be set via carbon.properties.</p> http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/src/main/webapp/sdk-guide.html ---------------------------------------------------------------------- diff --git a/src/main/webapp/sdk-guide.html b/src/main/webapp/sdk-guide.html index 08044d1..7128c6c 100644 --- a/src/main/webapp/sdk-guide.html +++ b/src/main/webapp/sdk-guide.html @@ -340,6 +340,14 @@ These SDK writer output contains just a carbondata and carbonindex files. No met </tbody> </table> <h2> +<a id="run-sql-on-files-directly" class="anchor" href="#run-sql-on-files-directly" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Run SQL on files directly</h2> +<p>Instead of creating table and query it, you can also query that file directly with SQL.</p> +<h3> +<a id="example" class="anchor" href="#example" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Example</h3> +<pre><code>SELECT * FROM carbonfile.`$Path` +</code></pre> +<p>Find example code at <a href="https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/DirectSQLExample.scala"; target=_blank>DirectSQLExample</a> in the CarbonData repo.</p> +<h2> <a id="api-list" class="anchor" href="#api-list" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>API List</h2> <h3> <a id="class-orgapachecarbondatasdkfilecarbonwriterbuilder" class="anchor" href="#class-orgapachecarbondatasdkfilecarbonwriterbuilder" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Class org.apache.carbondata.sdk.file.CarbonWriterBuilder</h3> @@ -578,17 +586,22 @@ External client can make use of this reader to read CarbonData files without Car String path = "./testWriteFiles"; CarbonReader reader = CarbonReader .builder(path, "_temp") - .projection(new String[]{"name", "age"}) + .projection(new String[]{"stringField", "shortField", "intField", "longField", + "doubleField", "boolField", "dateField", "timeField", "decimalField"}) .build(); // 2. Read data + long day = 24L * 3600 * 1000; int i = 0; while (reader.hasNext()) { - Object[] row = (Object[]) reader.readNextRow(); - System.out.println(row[0] + "\t" + row[1]); - i++; + Object[] row = (Object[]) reader.readNextRow(); + System.out.println(String.format("%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t", + i, row[0], row[1], row[2], row[3], row[4], row[5], + new Date((day * ((int) row[6]))), new Timestamp((long) row[7] / 1000), row[8] + )); + i++; } - + // 3. Close this reader reader.close(); </code></pre> @@ -597,12 +610,25 @@ External client can make use of this reader to read CarbonData files without Car <a id="api-list-1" class="anchor" href="#api-list-1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>API List</h2> <h3> <a id="class-orgapachecarbondatasdkfilecarbonreader" class="anchor" href="#class-orgapachecarbondatasdkfilecarbonreader" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Class org.apache.carbondata.sdk.file.CarbonReader</h3> -<pre><code> /** - * Return a new CarbonReaderBuilder instance - */ +<pre><code> /** + * Return a new {@link CarbonReaderBuilder} instance + * + * @param tablePath table store path + * @param tableName table name + * @return CarbonReaderBuilder object + */ public static CarbonReaderBuilder builder(String tablePath, String tableName); </code></pre> <pre><code> /** + * Return a new CarbonReaderBuilder instance + * Default value of table name is table + tablePath + time + * + * @param tablePath table path + * @return CarbonReaderBuilder object + */ + public static CarbonReaderBuilder builder(String tablePath); +</code></pre> +<pre><code> /** * Return true if has next row */ public boolean hasNext(); http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/src/main/webapp/timeseries-datamap-guide.html ---------------------------------------------------------------------- diff --git a/src/main/webapp/timeseries-datamap-guide.html b/src/main/webapp/timeseries-datamap-guide.html index b40150c..2400ce0 100644 --- a/src/main/webapp/timeseries-datamap-guide.html +++ b/src/main/webapp/timeseries-datamap-guide.html @@ -179,14 +179,14 @@ <div><h1> <a id="carbondata-timeseries-datamap" class="anchor" href="#carbondata-timeseries-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CarbonData Timeseries DataMap</h1> <ul> -<li><a href="#timeseries-datamap-intoduction-(alpha-feature-in-1.3.0)">Timeseries DataMap</a></li> +<li><a href="#timeseries-datamap-intoduction">Timeseries DataMap Introduction</a></li> <li><a href="#compacting-pre-aggregate-tables">Compaction</a></li> <li><a href="#data-management-with-pre-aggregate-tables">Data Management</a></li> </ul> <h2> -<a id="timeseries-datamap-intoduction-alpha-feature-in-130" class="anchor" href="#timeseries-datamap-intoduction-alpha-feature-in-130" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Timeseries DataMap Intoduction (Alpha feature in 1.3.0)</h2> -<p>Timeseries DataMap a pre-aggregate table implementation based on 'preaggregate' DataMap. -Difference is that Timerseries DataMap has built-in understanding of time hierarchy and +<a id="timeseries-datamap-introduction-alpha-feature-in-130" class="anchor" href="#timeseries-datamap-introduction-alpha-feature-in-130" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Timeseries DataMap Introduction (Alpha feature in 1.3.0)</h2> +<p>Timeseries DataMap a pre-aggregate table implementation based on 'pre-aggregate' DataMap. +Difference is that Timeseries DataMap has built-in understanding of time hierarchy and levels: year, month, day, hour, minute, so that it supports automatic roll-up in time dimension for query.</p> <p>The data loading, querying, compaction command and its behavior is the same as preaggregate DataMap. http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/src/site/markdown/sdk-guide.md ---------------------------------------------------------------------- diff --git a/src/site/markdown/sdk-guide.md b/src/site/markdown/sdk-guide.md index 1d225a9..2371b33 100644 --- a/src/site/markdown/sdk-guide.md +++ b/src/site/markdown/sdk-guide.md @@ -128,7 +128,14 @@ Each of SQL data types are mapped into data types of SDK. Following are the mapp | STRING | DataTypes.STRING | | DECIMAL | DataTypes.createDecimalType(precision, scale) | +## Run SQL on files directly +Instead of creating table and query it, you can also query that file directly with SQL. +### Example +``` +SELECT * FROM carbonfile.`$Path` +``` +Find example code at [DirectSQLExample](https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/DirectSQLExample.scala) in the CarbonData repo. ## API List ### Class org.apache.carbondata.sdk.file.CarbonWriterBuilder @@ -408,17 +415,22 @@ External client can make use of this reader to read CarbonData files without Car String path = "./testWriteFiles"; CarbonReader reader = CarbonReader .builder(path, "_temp") - .projection(new String[]{"name", "age"}) + .projection(new String[]{"stringField", "shortField", "intField", "longField", + "doubleField", "boolField", "dateField", "timeField", "decimalField"}) .build(); // 2. Read data + long day = 24L * 3600 * 1000; int i = 0; while (reader.hasNext()) { - Object[] row = (Object[]) reader.readNextRow(); - System.out.println(row[0] + "\t" + row[1]); - i++; + Object[] row = (Object[]) reader.readNextRow(); + System.out.println(String.format("%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t", + i, row[0], row[1], row[2], row[3], row[4], row[5], + new Date((day * ((int) row[6]))), new Timestamp((long) row[7] / 1000), row[8] + )); + i++; } - + // 3. Close this reader reader.close(); ``` @@ -429,14 +441,29 @@ Find example code at [CarbonReaderExample](https://github.com/apache/carbondata/ ### Class org.apache.carbondata.sdk.file.CarbonReader ``` - /** - * Return a new CarbonReaderBuilder instance - */ + /** + * Return a new {@link CarbonReaderBuilder} instance + * + * @param tablePath table store path + * @param tableName table name + * @return CarbonReaderBuilder object + */ public static CarbonReaderBuilder builder(String tablePath, String tableName); ``` ``` /** + * Return a new CarbonReaderBuilder instance + * Default value of table name is table + tablePath + time + * + * @param tablePath table path + * @return CarbonReaderBuilder object + */ + public static CarbonReaderBuilder builder(String tablePath); +``` + +``` + /** * Return true if has next row */ public boolean hasNext();
