Repository: arrow-site Updated Branches: refs/heads/asf-site 796ce23f3 -> c43352140
Update for 0.5.0 Project: http://git-wip-us.apache.org/repos/asf/arrow-site/repo Commit: http://git-wip-us.apache.org/repos/asf/arrow-site/commit/c4335214 Tree: http://git-wip-us.apache.org/repos/asf/arrow-site/tree/c4335214 Diff: http://git-wip-us.apache.org/repos/asf/arrow-site/diff/c4335214 Branch: refs/heads/asf-site Commit: c43352140fc3100e8b7109409104db9a5334000b Parents: 796ce23 Author: Wes McKinney <[email protected]> Authored: Tue Jul 25 10:39:47 2017 -0400 Committer: Wes McKinney <[email protected]> Committed: Tue Jul 25 10:39:47 2017 -0400 ---------------------------------------------------------------------- blog/2017/07/25/0.5.0-release/index.html | 223 +++++++++++++++++++ blog/index.html | 115 ++++++++++ docs/memory_layout.html | 31 +-- feed.xml | 93 +++++++- index.html | 16 +- install/index.html | 26 ++- release/0.5.0.html | 300 ++++++++++++++++++++++++++ release/index.html | 1 + 8 files changed, 767 insertions(+), 38 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/arrow-site/blob/c4335214/blog/2017/07/25/0.5.0-release/index.html ---------------------------------------------------------------------- diff --git a/blog/2017/07/25/0.5.0-release/index.html b/blog/2017/07/25/0.5.0-release/index.html new file mode 100644 index 0000000..0dc3f80 --- /dev/null +++ b/blog/2017/07/25/0.5.0-release/index.html @@ -0,0 +1,223 @@ +<!DOCTYPE html> +<html lang="en-US"> + <head> + <meta charset="UTF-8"> + <title>Apache Arrow 0.5.0 Release</title> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <meta name="generator" content="Jekyll v3.4.3"> + <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> + <link rel="icon" type="image/x-icon" href="/favicon.ico"> + + <title>Apache Arrow Homepage</title> + <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900"> + + <link href="/css/main.css" rel="stylesheet"> + <link href="/css/syntax.css" rel="stylesheet"> + <script src="https://code.jquery.com/jquery-3.2.1.min.js"; + integrity="sha256-hwg4gsxgFZhOsEEamdOYGBf13FyQuiTwlAQgxVSNgt4=" + crossorigin="anonymous"></script> + <script src="/assets/javascripts/bootstrap.min.js"></script> + </head> + + + +<body class="wrap"> + <div class="container"> + <nav class="navbar navbar-default"> + <div class="container-fluid"> + <div class="navbar-header"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#arrow-navbar"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="navbar-brand" href="/">Apache Arrow™ </a> + </div> + + <!-- Collect the nav links, forms, and other content for toggling --> + <div class="collapse navbar-collapse" id="arrow-navbar"> + <ul class="nav navbar-nav"> + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown" + role="button" aria-haspopup="true" + aria-expanded="false">Project Links<span class="caret"></span> + </a> + <ul class="dropdown-menu"> + <li><a href="/install/">Install</a></li> + <li><a href="/blog/">Blog</a></li> + <li><a href="/release/">Releases</a></li> + <li><a href="https://issues.apache.org/jira/browse/ARROW";>Issue Tracker</a></li> + <li><a href="https://github.com/apache/arrow";>Source Code</a></li> + <li><a href="http://mail-archives.apache.org/mod_mbox/arrow-dev/";>Mailing List</a></li> + <li><a href="https://apachearrowslackin.herokuapp.com";>Slack Channel</a></li> + <li><a href="/committers/">Committers</a></li> + </ul> + </li> + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown" + role="button" aria-haspopup="true" + aria-expanded="false">Specification<span class="caret"></span> + </a> + <ul class="dropdown-menu"> + <li><a href="/docs/memory_layout.html">Memory Layout</a></li> + <li><a href="/docs/metadata.html">Metadata</a></li> + <li><a href="/docs/ipc.html">Messaging / IPC</a></li> + </ul> + </li> + + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown" + role="button" aria-haspopup="true" + aria-expanded="false">Documentation<span class="caret"></span> + </a> + <ul class="dropdown-menu"> + <li><a href="/docs/python">Python</a></li> + <li><a href="/docs/cpp">C++ API</a></li> + <li><a href="/docs/java">Java API</a></li> + <li><a href="/docs/c_glib">C GLib API</a></li> + </ul> + </li> + <!-- <li><a href="/blog">Blog</a></li> --> + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown" + role="button" aria-haspopup="true" + aria-expanded="false">ASF Links<span class="caret"></span> + </a> + <ul class="dropdown-menu"> + <li><a href="http://www.apache.org/";>ASF Website</a></li> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Donate</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </li> + </ul> + <a href="http://www.apache.org/";> + <img style="float:right;" src="/img/asf_logo.svg" width="120px"/> + </a> + </div><!-- /.navbar-collapse --> + </div> + </nav> + + + <h2> + Apache Arrow 0.5.0 Release + <a href="/blog/2017/07/25/0.5.0-release/" class="permalink" title="Permalink">â</a> + </h2> + + + + <div class="panel"> + <div class="panel-body"> + <div> + <span class="label label-default">Published</span> + <span class="published"> + <i class="fa fa-calendar"></i> + 25 Jul 2017 + </span> + </div> + <div> + <span class="label label-default">By</span> + <a href="http://wesmckinney.com";><i class="fa fa-user"></i> Wes McKinney (wesm)</a> + </div> + </div> + </div> + + <!-- + +--> + +<p>The Apache Arrow team is pleased to announce the 0.5.0 release. It includes +<a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.5.0"><strong>130 resolved JIRAs</strong></a> with some new features, expanded integration +testing between implementations, and bug fixes. The Arrow memory format remains +stable since the 0.3.x and 0.4.x releases.</p> + +<p>See the <a href="http://arrow.apache.org/install";>Install Page</a> to learn how to get the libraries for your +platform. The <a href="http://arrow.apache.org/release/0.5.0.html";>complete changelog</a> is also available.</p> + +<h2 id="expanded-integration-testing">Expanded Integration Testing</h2> + +<p>In this release, we added compatibility tests for dictionary-encoded data +between Java and C++. This enables the distinct values (the <em>dictionary</em>) in a +vector to be transmitted as part of an Arrow schema while the record batches +contain integers which correspond to the dictionary.</p> + +<p>So we might have:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>data (string): ['foo', 'bar', 'foo', 'bar'] +</code></pre> +</div> + +<p>In dictionary-encoded form, this could be represented as:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>indices (int8): [0, 1, 0, 1] +dictionary (string): ['foo', 'bar'] +</code></pre> +</div> + +<p>In upcoming releases, we plan to complete integration testing for the remaining +data types (including some more complicated types like unions and decimals) on +the road to a 1.0.0 release in the future.</p> + +<h2 id="c-activity">C++ Activity</h2> + +<p>We completed a number of significant pieces of work in the C++ part of Apache +Arrow.</p> + +<h3 id="using-jemalloc-as-default-memory-allocator">Using jemalloc as default memory allocator</h3> + +<p>We decided to use <a href="https://github.com/jemalloc/jemalloc";>jemalloc</a> as the default memory allocator unless it is +explicitly disabled. This memory allocator has significant performance +advantages in Arrow workloads over the default <code class="highlighter-rouge">malloc</code> implementation. We will +publish a blog post going into more detail about this and why you might care.</p> + +<h3 id="sharing-more-c-code-with-apache-parquet">Sharing more C++ code with Apache Parquet</h3> + +<p>We imported the compression library interfaces and dictionary encoding +algorithms from the <a href="http://github.com/apache/parquet-cpp";>Apache Parquet C++ library</a>. The Parquet library now +depends on this code in Arrow, and we will be able to use it more easily for +data compression in Arrow use cases.</p> + +<p>As part of incorporating Parquetâs dictionary encoding utilities, we have +developed an <code class="highlighter-rouge">arrow::DictionaryBuilder</code> class to enable building +dictionary-encoded arrays iteratively. This can help save memory and yield +better performance when interacting with databases, Parquet files, or other +sources which may have columns having many duplicates.</p> + +<h3 id="support-for-lz4-and-zstd-compressors">Support for LZ4 and ZSTD compressors</h3> + +<p>We added LZ4 and ZSTD compression library support. In ARROW-300 and other +planned work, we intend to add some compression features for data sent via RPC.</p> + +<h2 id="python-activity">Python Activity</h2> + +<p>We fixed many bugs which were affecting Parquet and Feather users and fixed +several other rough edges with normal Arrow use. We also added some additional +Arrow type conversions: structs, lists embedded in pandas objects, and Arrow +time types (which deserialize to the <code class="highlighter-rouge">datetime.time</code> type).</p> + +<p>In upcoming releases we plan to continue to improve <a href="http://github.com/dask/dask";>Dask</a> support and +performance for distributed processing of Apache Parquet files with pyarrow.</p> + +<h2 id="the-road-ahead">The Road Ahead</h2> + +<p>We have much work ahead of us to build out Arrow integrations in other data +systems to improve their processing performance and interoperability with other +systems.</p> + +<p>We are discussing the roadmap to a future 1.0.0 release on the <a href="http://mail-archives.apache.org/mod_mbox/arrow-dev/";>developer +mailing list</a>. Please join the discussion there.</p> + + + + <hr/> +<footer class="footer"> + <p>Apache Arrow, Arrow, Apache, the Apache feather logo, and the Apache Arrow project logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.</p> + <p>© 2017 Apache Software Foundation</p> +</footer> + + </div> +</body> +</html> http://git-wip-us.apache.org/repos/asf/arrow-site/blob/c4335214/blog/index.html ---------------------------------------------------------------------- diff --git a/blog/index.html b/blog/index.html index 9b2c972..c8a810a 100644 --- a/blog/index.html +++ b/blog/index.html @@ -111,6 +111,121 @@ <div class="container"> <h2> + Apache Arrow 0.5.0 Release + <a href="/blog/2017/07/25/0.5.0-release/" class="permalink" title="Permalink">â</a> + </h2> + + + + <div class="panel"> + <div class="panel-body"> + <div> + <span class="label label-default">Published</span> + <span class="published"> + <i class="fa fa-calendar"></i> + 25 Jul 2017 + </span> + </div> + <div> + <span class="label label-default">By</span> + <a href="http://wesmckinney.com";><i class="fa fa-user"></i> Wes McKinney (wesm)</a> + </div> + </div> + </div> + <!-- + +--> + +<p>The Apache Arrow team is pleased to announce the 0.5.0 release. It includes +<a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.5.0"><strong>130 resolved JIRAs</strong></a> with some new features, expanded integration +testing between implementations, and bug fixes. The Arrow memory format remains +stable since the 0.3.x and 0.4.x releases.</p> + +<p>See the <a href="http://arrow.apache.org/install";>Install Page</a> to learn how to get the libraries for your +platform. The <a href="http://arrow.apache.org/release/0.5.0.html";>complete changelog</a> is also available.</p> + +<h2 id="expanded-integration-testing">Expanded Integration Testing</h2> + +<p>In this release, we added compatibility tests for dictionary-encoded data +between Java and C++. This enables the distinct values (the <em>dictionary</em>) in a +vector to be transmitted as part of an Arrow schema while the record batches +contain integers which correspond to the dictionary.</p> + +<p>So we might have:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>data (string): ['foo', 'bar', 'foo', 'bar'] +</code></pre> +</div> + +<p>In dictionary-encoded form, this could be represented as:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>indices (int8): [0, 1, 0, 1] +dictionary (string): ['foo', 'bar'] +</code></pre> +</div> + +<p>In upcoming releases, we plan to complete integration testing for the remaining +data types (including some more complicated types like unions and decimals) on +the road to a 1.0.0 release in the future.</p> + +<h2 id="c-activity">C++ Activity</h2> + +<p>We completed a number of significant pieces of work in the C++ part of Apache +Arrow.</p> + +<h3 id="using-jemalloc-as-default-memory-allocator">Using jemalloc as default memory allocator</h3> + +<p>We decided to use <a href="https://github.com/jemalloc/jemalloc";>jemalloc</a> as the default memory allocator unless it is +explicitly disabled. This memory allocator has significant performance +advantages in Arrow workloads over the default <code class="highlighter-rouge">malloc</code> implementation. We will +publish a blog post going into more detail about this and why you might care.</p> + +<h3 id="sharing-more-c-code-with-apache-parquet">Sharing more C++ code with Apache Parquet</h3> + +<p>We imported the compression library interfaces and dictionary encoding +algorithms from the <a href="http://github.com/apache/parquet-cpp";>Apache Parquet C++ library</a>. The Parquet library now +depends on this code in Arrow, and we will be able to use it more easily for +data compression in Arrow use cases.</p> + +<p>As part of incorporating Parquetâs dictionary encoding utilities, we have +developed an <code class="highlighter-rouge">arrow::DictionaryBuilder</code> class to enable building +dictionary-encoded arrays iteratively. This can help save memory and yield +better performance when interacting with databases, Parquet files, or other +sources which may have columns having many duplicates.</p> + +<h3 id="support-for-lz4-and-zstd-compressors">Support for LZ4 and ZSTD compressors</h3> + +<p>We added LZ4 and ZSTD compression library support. In ARROW-300 and other +planned work, we intend to add some compression features for data sent via RPC.</p> + +<h2 id="python-activity">Python Activity</h2> + +<p>We fixed many bugs which were affecting Parquet and Feather users and fixed +several other rough edges with normal Arrow use. We also added some additional +Arrow type conversions: structs, lists embedded in pandas objects, and Arrow +time types (which deserialize to the <code class="highlighter-rouge">datetime.time</code> type).</p> + +<p>In upcoming releases we plan to continue to improve <a href="http://github.com/dask/dask";>Dask</a> support and +performance for distributed processing of Apache Parquet files with pyarrow.</p> + +<h2 id="the-road-ahead">The Road Ahead</h2> + +<p>We have much work ahead of us to build out Arrow integrations in other data +systems to improve their processing performance and interoperability with other +systems.</p> + +<p>We are discussing the roadmap to a future 1.0.0 release on the <a href="http://mail-archives.apache.org/mod_mbox/arrow-dev/";>developer +mailing list</a>. Please join the discussion there.</p> + + + </div> + + + + + + <div class="container"> + <h2> Connecting Relational Databases to the Apache Arrow World with turbodbc <a href="/blog/2017/06/16/turbodbc-arrow/" class="permalink" title="Permalink">â</a> </h2> http://git-wip-us.apache.org/repos/asf/arrow-site/blob/c4335214/docs/memory_layout.html ---------------------------------------------------------------------- diff --git a/docs/memory_layout.html b/docs/memory_layout.html index 32c5b92..7703a15 100644 --- a/docs/memory_layout.html +++ b/docs/memory_layout.html @@ -503,37 +503,42 @@ primitive value array having Int32 logical type.</char></p> * Children arrays: * field-0 array (`List<char>`): - * Length: 4, Null count: 1 + * Length: 4, Null count: 2 * Null bitmap buffer: | Byte 0 (validity bitmap) | Bytes 1-63 | |--------------------------|-----------------------| - | 00001101 | 0 (padding) | + | 00001001 | 0 (padding) | * Offsets buffer: | Bytes 0-19 | |----------------| - | 0, 3, 3, 6, 10 | + | 0, 3, 3, 3, 7 | * Values array: - * Length: 10, Null count: 0 + * Length: 7, Null count: 0 * Null bitmap buffer: Not required * Value buffer: - | Bytes 0-9 | + | Bytes 0-6 | |----------------| - | joebobmark | + | joemark | * field-1 array (int32 array): - * Length: 4, Null count: 0 - * Null bitmap buffer: Not required + * Length: 4, Null count: 1 + * Null bitmap buffer: + + | Byte 0 (validity bitmap) | Bytes 1-63 | + |--------------------------|-----------------------| + | 00001011 | 0 (padding) | + * Value Buffer: - | Bytes 0-15 | - |----------------| - | 1, 2, 3, 4 | + |Bytes 0-3 | Bytes 4-7 | Bytes 8-11 | Bytes 12-15 | Bytes 16-63 | + |------------|-------------|-------------|-------------|-------------| + | 1 | 2 | unspecified | 4 | unspecified | </code></pre> </div> @@ -670,7 +675,7 @@ union, it has some advantages that may be desirable in certain use cases:</p> |Bytes 0-3 | Bytes 4-7 | Bytes 8-11 | Bytes 12-15 | Bytes 16-19 | Bytes 20-23 | Bytes 24-63 | |------------|-------------|-------------|-------------|-------------|--------------|-----------------------| - | 1 | unspecified | unspecified | unspecified | 4 | unspecified | unspecified (padding) | + | 5 | unspecified | unspecified | unspecified | 4 | unspecified | unspecified (padding) | * u1 (float): * Length: 6, Null count: 4 @@ -724,7 +729,7 @@ the the types array indicates that a slot contains a different type at the index <h2 id="dictionary-encoding">Dictionary encoding</h2> <p>When a field is dictionary encoded, the values are represented by an array of Int32 representing the index of the value in the dictionary. -The Dictionary is received as a DictionaryBacth whose id is referenced by a dictionary attribute defined in the metadata (Message.fbs) in the Field table. +The Dictionary is received as a DictionaryBatch whose id is referenced by a dictionary attribute defined in the metadata (<a href="https://github.com/apache/arrow/blob/master/format/Message.fbs";>Message.fbs</a>) in the Field table. The dictionary has the same layout as the type of the field would dictate. Each entry in the dictionary can be accessed by its index in the DictionaryBatch. When a Schema references a Dictionary id, it must send a DictionaryBatch for this id before any RecordBatch.</p> http://git-wip-us.apache.org/repos/asf/arrow-site/blob/c4335214/feed.xml ---------------------------------------------------------------------- diff --git a/feed.xml b/feed.xml index fadbf40..1bbcf0e 100644 --- a/feed.xml +++ b/feed.xml @@ -1,4 +1,87 @@ -<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom"; ><generator uri="https://jekyllrb.com/"; version="3.4.3">Jekyll</generator><link href="/feed.xml" rel="self" type="application/atom+xml" /><link href="/" rel="alternate" type="text/html" /><updated>2017-06-27T11:53:04+02:00</updated><id>/</id><entry><title type="html">Connecting Relational Databases to the Apache Arrow World with turbodbc</title><link href="/blog/2017/06/16/turbodbc-arrow/" rel="alternate" type="text/html" title="Connecting Relational Databases to the Apache Arrow World with turbodbc" /><published>2017-06-16T10:00:00+02:00</published><updated>2017-06-16T10:00:00+02:00</updated><id>/blog/2017/06/16/turbodbc-arrow</id><content type="html" xml:base="/blog/2017/06/16/turbodbc-arrow/"><!-- +<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom"; ><generator uri="https://jekyllrb.com/"; version="3.4.3">Jekyll</generator><link href="/feed.xml" rel="self" type="application/atom+xml" /><link href="/" rel="alternate" type="text/html" /><updated>2017-07-25T10:39:23-04:00</updated><id>/</id><entry><title type="html">Apache Arrow 0.5.0 Release</title><link href="/blog/2017/07/25/0.5.0-release/" rel="alternate" type="text/html" title="Apache Arrow 0.5.0 Release" /><published>2017-07-25T00:00:00-04:00</published><updated>2017-07-25T00:00:00-04:00</updated><id>/blog/2017/07/25/0.5.0-release</id><content type="html" xml:base="/blog/2017/07/25/0.5.0-release/"><!-- + +--> + +<p>The Apache Arrow team is pleased to announce the 0.5.0 release. It includes +<a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.5.0"><strong>130 resolved JIRAs</strong></a> with some new features, expanded integration +testing between implementations, and bug fixes. The Arrow memory format remains +stable since the 0.3.x and 0.4.x releases.</p> + +<p>See the <a href="http://arrow.apache.org/install">Install Page</a> to learn how to get the libraries for your +platform. The <a href="http://arrow.apache.org/release/0.5.0.html">complete changelog</a> is also available.</p> + +<h2 id="expanded-integration-testing">Expanded Integration Testing</h2> + +<p>In this release, we added compatibility tests for dictionary-encoded data +between Java and C++. This enables the distinct values (the <em>dictionary</em>) in a +vector to be transmitted as part of an Arrow schema while the record batches +contain integers which correspond to the dictionary.</p> + +<p>So we might have:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>data (string): ['foo', 'bar', 'foo', 'bar'] +</code></pre> +</div> + +<p>In dictionary-encoded form, this could be represented as:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>indices (int8): [0, 1, 0, 1] +dictionary (string): ['foo', 'bar'] +</code></pre> +</div> + +<p>In upcoming releases, we plan to complete integration testing for the remaining +data types (including some more complicated types like unions and decimals) on +the road to a 1.0.0 release in the future.</p> + +<h2 id="c-activity">C++ Activity</h2> + +<p>We completed a number of significant pieces of work in the C++ part of Apache +Arrow.</p> + +<h3 id="using-jemalloc-as-default-memory-allocator">Using jemalloc as default memory allocator</h3> + +<p>We decided to use <a href="https://github.com/jemalloc/jemalloc">jemalloc</a> as the default memory allocator unless it is +explicitly disabled. This memory allocator has significant performance +advantages in Arrow workloads over the default <code class="highlighter-rouge">malloc</code> implementation. We will +publish a blog post going into more detail about this and why you might care.</p> + +<h3 id="sharing-more-c-code-with-apache-parquet">Sharing more C++ code with Apache Parquet</h3> + +<p>We imported the compression library interfaces and dictionary encoding +algorithms from the <a href="http://github.com/apache/parquet-cpp">Apache Parquet C++ library</a>. The Parquet library now +depends on this code in Arrow, and we will be able to use it more easily for +data compression in Arrow use cases.</p> + +<p>As part of incorporating Parquetâs dictionary encoding utilities, we have +developed an <code class="highlighter-rouge">arrow::DictionaryBuilder</code> class to enable building +dictionary-encoded arrays iteratively. This can help save memory and yield +better performance when interacting with databases, Parquet files, or other +sources which may have columns having many duplicates.</p> + +<h3 id="support-for-lz4-and-zstd-compressors">Support for LZ4 and ZSTD compressors</h3> + +<p>We added LZ4 and ZSTD compression library support. In ARROW-300 and other +planned work, we intend to add some compression features for data sent via RPC.</p> + +<h2 id="python-activity">Python Activity</h2> + +<p>We fixed many bugs which were affecting Parquet and Feather users and fixed +several other rough edges with normal Arrow use. We also added some additional +Arrow type conversions: structs, lists embedded in pandas objects, and Arrow +time types (which deserialize to the <code class="highlighter-rouge">datetime.time</code> type).</p> + +<p>In upcoming releases we plan to continue to improve <a href="http://github.com/dask/dask">Dask</a> support and +performance for distributed processing of Apache Parquet files with pyarrow.</p> + +<h2 id="the-road-ahead">The Road Ahead</h2> + +<p>We have much work ahead of us to build out Arrow integrations in other data +systems to improve their processing performance and interoperability with other +systems.</p> + +<p>We are discussing the roadmap to a future 1.0.0 release on the <a href="http://mail-archives.apache.org/mod_mbox/arrow-dev/">developer +mailing list</a>. Please join the discussion there.</p></content><author><name>wesm</name></author></entry><entry><title type="html">Connecting Relational Databases to the Apache Arrow World with turbodbc</title><link href="/blog/2017/06/16/turbodbc-arrow/" rel="alternate" type="text/html" title="Connecting Relational Databases to the Apache Arrow World with turbodbc" /><published>2017-06-16T04:00:00-04:00</published><updated>2017-06-16T04:00:00-04:00</updated><id>/blog/2017/06/16/turbodbc-arrow</id><content type="html" xml:base="/blog/2017/06/16/turbodbc-arrow/"><!-- --> @@ -77,7 +160,7 @@ databases.</p> <p>If you would like to learn more about turbodbc, check out the <a href="https://github.com/blue-yonder/turbodbc">GitHub project</a> and the <a href="http://turbodbc.readthedocs.io/">project documentation</a>. If you want to learn more about how turbodbc implements the nitty-gritty details, check out parts <a href="https://tech.blue-yonder.com/making-of-turbodbc-part-1-wrestling-with-the-side-effects-of-a-c-api/">one</a> and <a href="https://tech.blue-yonder.com/making-of-turbodbc-part-2-c-to-python/">two</a> of the -<a href="https://tech.blue-yonder.com/making-of-turbodbc-part-1-wrestling-with-the-side-effects-of-a-c-api/">âMaking of turbodbcâ</a> series at <a href="https://tech.blue-yonder.com/">Blue Yonderâs technology blog</a>.</p></content><author><name>MathMagique</name></author></entry><entry><title type="html">Apache Arrow 0.4.1 Release</title><link href="/blog/2017/06/14/0.4.1-release/" rel="alternate" type="text/html" title="Apache Arrow 0.4.1 Release" /><published>2017-06-14T16:00:00+02:00</published><updated>2017-06-14T16:00:00+02:00</updated><id>/blog/2017/06/14/0.4.1-release</id><content type="html" xml:base="/blog/2017/06/14/0.4.1-release/"><!-- +<a href="https://tech.blue-yonder.com/making-of-turbodbc-part-1-wrestling-with-the-side-effects-of-a-c-api/">âMaking of turbodbcâ</a> series at <a href="https://tech.blue-yonder.com/">Blue Yonderâs technology blog</a>.</p></content><author><name>MathMagique</name></author></entry><entry><title type="html">Apache Arrow 0.4.1 Release</title><link href="/blog/2017/06/14/0.4.1-release/" rel="alternate" type="text/html" title="Apache Arrow 0.4.1 Release" /><published>2017-06-14T10:00:00-04:00</published><updated>2017-06-14T10:00:00-04:00</updated><id>/blog/2017/06/14/0.4.1-release</id><content type="html" xml:base="/blog/2017/06/14/0.4.1-release/"><!-- --> @@ -112,7 +195,7 @@ team used the PyArrow C++ API introduced in version 0.4.0 to construct <div class="highlighter-rouge"><pre class="highlight"><code>pip install turbodbc conda install turbodbc -c conda-forge </code></pre> -</div></content><author><name>wesm</name></author></entry><entry><title type="html">Apache Arrow 0.4.0 Release</title><link href="/blog/2017/05/23/0.4.0-release/" rel="alternate" type="text/html" title="Apache Arrow 0.4.0 Release" /><published>2017-05-23T06:00:00+02:00</published><updated>2017-05-23T06:00:00+02:00</updated><id>/blog/2017/05/23/0.4.0-release</id><content type="html" xml:base="/blog/2017/05/23/0.4.0-release/"><!-- +</div></content><author><name>wesm</name></author></entry><entry><title type="html">Apache Arrow 0.4.0 Release</title><link href="/blog/2017/05/23/0.4.0-release/" rel="alternate" type="text/html" title="Apache Arrow 0.4.0 Release" /><published>2017-05-23T00:00:00-04:00</published><updated>2017-05-23T00:00:00-04:00</updated><id>/blog/2017/05/23/0.4.0-release</id><content type="html" xml:base="/blog/2017/05/23/0.4.0-release/"><!-- --> @@ -185,7 +268,7 @@ pyarrow.import_pyarrow()</code></pre></figure> <h3 id="python-wheel-installers-on-macos">Python Wheel Installers on macOS</h3> <p>With this release, <code class="highlighter-rouge">pip install pyarrow</code> works on macOS (OS X) as well as -Linux. We are working on providing binary wheel installers for Windows as well.</p></content><author><name>wesm</name></author></entry><entry><title type="html">Apache Arrow 0.3.0 Release</title><link href="/blog/2017/05/08/0.3-release/" rel="alternate" type="text/html" title="Apache Arrow 0.3.0 Release" /><published>2017-05-08T06:00:00+02:00</published><updated>2017-05-08T06:00:00+02:00</updated><id>/blog/2017/05/08/0.3-release</id><content type="html" xml:base="/blog/2017/05/08/0.3-release/"><!-- +Linux. We are working on providing binary wheel installers for Windows as well.</p></content><author><name>wesm</name></author></entry><entry><title type="html">Apache Arrow 0.3.0 Release</title><link href="/blog/2017/05/08/0.3-release/" rel="alternate" type="text/html" title="Apache Arrow 0.3.0 Release" /><published>2017-05-08T00:00:00-04:00</published><updated>2017-05-08T00:00:00-04:00</updated><id>/blog/2017/05/08/0.3-release</id><content type="html" xml:base="/blog/2017/05/08/0.3-release/"><!-- --> @@ -397,7 +480,7 @@ instructions for getting started.</p> 1 Holden Karau 1 Deepak Majeti </code></pre> -</div></content><author><name>wesm</name></author></entry><entry><title type="html">Apache Arrow 0.3.0ãªãªã¼ã¹</title><link href="/blog/2017/05/08/0.3-release-japanese/" rel="alternate" type="text/html" title="Apache Arrow 0.3.0ãªãªã¼ã¹" /><published>2017-05-08T06:00:00+02:00</published><updated>2017-05-08T06:00:00+02:00</updated><id>/blog/2017/05/08/0.3-release-japanese</id><content type="html" xml:base="/blog/2017/05/08/0.3-release-japanese/"><!-- +</div></content><author><name>wesm</name></author></entry><entry><title type="html">Apache Arrow 0.3.0ãªãªã¼ã¹</title><link href="/blog/2017/05/08/0.3-release-japanese/" rel="alternate" type="text/html" title="Apache Arrow 0.3.0ãªãªã¼ã¹" /><published>2017-05-08T00:00:00-04:00</published><updated>2017-05-08T00:00:00-04:00</updated><id>/blog/2017/05/08/0.3-release-japanese</id><content type="html" xml:base="/blog/2017/05/08/0.3-release-japanese/"><!-- --> http://git-wip-us.apache.org/repos/asf/arrow-site/blob/c4335214/index.html ---------------------------------------------------------------------- diff --git a/index.html b/index.html index 9b61b13..95c1f23 100644 --- a/index.html +++ b/index.html @@ -107,14 +107,18 @@ <p class="lead">Powering Columnar In-Memory Analytics</p> <p> <a class="btn btn-lg btn-success" href="mailto:[email protected]"; role="button">Join Mailing List</a> - <a class="btn btn-lg btn-primary" href="/install/" role="button">Install (0.4.1 Release - June 9, 2017)</a> + <a class="btn btn-lg btn-primary" href="/install/" role="button">Install (0.5.0 Release - July 23, 2017)</a> </p> </div> - <h4><strong>Latest News</strong>: <a href="/blog/">Apache Arrow 0.4.1 release</a></h4> + <h4><strong>Latest News</strong>: <a href="/blog/">Apache Arrow 0.5.0 release</a></h4> <div class="row"> <div class="col-lg-4"> <h2>Fast</h2> - <p>Apache Arrow™ enables execution engines to take advantage of the latest SIMD (Single input multiple data) operations included in modern processors, for native vectorized optimization of analytical data processing. Columnar layout of data also allows for a better use of CPU caches by placing all data relevant to a column operation in as compact of a format as possible.</p> + <p>Apache Arrow™ enables execution engines to take advantage of the latest SIM +D (Single input multiple data) operations included in modern processors, for native vectorized optimization of analytical data processing. Columnar layout of data also allows for a better use of CPU caches by placing all data relevant to a column operation in as compact of a format + as possible.</p> + <p>The Arrow memory format supports <strong>zero-copy reads</strong> + for lightning-fast data access without serialization overhead.</p> </div> <div class="col-lg-4"> <h2>Flexible</h2> @@ -126,12 +130,6 @@ </div> </div> <!-- close "row" div --> -<h2>Zero-Copy IPC and Streaming Messaging</h2> -<div align="left"> - <h4>Apache Arrow supports zero-copy shared memory IPC and a streaming wire - format that fully avoids traditional data serialization costs</h4> -</div> - <h2>Performance Advantage of Columnar In-Memory</h2> <div align="center"> <img src="img/simd.png" alt="SIMD" style="width:60%" /> http://git-wip-us.apache.org/repos/asf/arrow-site/blob/c4335214/install/index.html ---------------------------------------------------------------------- diff --git a/install/index.html b/install/index.html index ec70f25..0b42f41 100644 --- a/install/index.html +++ b/install/index.html @@ -105,23 +105,27 @@ --> -<h2 id="current-version-041">Current Version: 0.4.1</h2> +<h2 id="current-version-050">Current Version: 0.5.0</h2> -<h3 id="released-9-june-2017">Released: 9 June 2017</h3> +<h3 id="released-23-july-2017">Released: 23 July 2017</h3> -<p>See the <a href="http://arrow.apache.org/release/0.4.1.html";>release notes</a> and <a href="http://arrow.apache.org/blog/2017/06/14/0.4.1-release/";>blog post</a> for more about whatâs new.</p> +<p>See the <a href="http://arrow.apache.org/release/0.5.0.html";>release notes</a> and <a href="http://arrow.apache.org/blog/2017/07/24/0.5.0-release/";>blog post</a> for more about whatâs new.</p> <h3 id="source-release">Source release</h3> <ul> - <li><strong>Source Release</strong>: <a href="https://www.apache.org/dyn/closer.cgi/arrow/arrow-0.4.1/apache-arrow-0.4.1.tar.gz";>apache-arrow-0.4.1.tar.gz</a></li> - <li><strong>Verification</strong>: <a href="https://www.apache.org/dyn/closer.cgi/arrow/arrow-0.4.1/apache-arrow-0.4.1.tar.gz.md5";>md5</a>, <a href="https://www.apache.org/dyn/closer.cgi/arrow/arrow-0.4.1/apache-arrow-0.4.1.tar.gz.asc";>asc</a></li> - <li><a href="https://github.com/apache/arrow/releases/tag/apache-arrow-0.4.1";>Git tag 46315431</a></li> + <li><strong>Source Release</strong>: <a href="https://www.apache.org/dyn/closer.cgi/arrow/arrow-0.5.0/apache-arrow-0.5.0.tar.gz";>apache-arrow-0.5.0.tar.gz</a></li> + <li><strong>Verification</strong>: <a href="https://www.apache.org/dyn/closer.cgi/arrow/arrow-0.5.0/apache-arrow-0.5.0.tar.gz.md5";>md5</a>, <a href="https://www.apache.org/dyn/closer.cgi/arrow/arrow-0.5.0/apache-arrow-0.5.0.tar.gz.asc";>asc</a></li> + <li><a href="https://github.com/apache/arrow/releases/tag/apache-arrow-0.5.0";>Git tag e9f76e1</a></li> </ul> <h3 id="java-packages">Java Packages</h3> -<p><a href="http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.arrow%22%20AND%20v%3A%220.4.1%22";>Java Artifacts on Maven Central</a></p> +<p><a href="http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.arrow%22%20AND%20v%3A%220.5.0%22";>Java Artifacts on Maven Central</a></p> + +<h2 id="binary-installers-for-c-c-python">Binary Installers for C, C++, Python</h2> + +<p>It may take a little time for the binary packages to get updated</p> <h3 id="c-and-python-conda-packages-unofficial">C++ and Python Conda Packages (Unofficial)</h3> @@ -129,14 +133,14 @@ platforms:</p> <ul> - <li>Linux and OS X (Python 2.7, 3.5, and 3.6)</li> + <li>Linux and macOS (Python 2.7, 3.5, and 3.6)</li> <li>Windows (Python 3.5 and 3.6)</li> </ul> <p>Install them with:</p> -<div class="language-shell highlighter-rouge"><pre class="highlight"><code>conda install arrow-cpp -c conda-forge -conda install pyarrow -c conda-forge +<div class="language-shell highlighter-rouge"><pre class="highlight"><code>conda install arrow-cpp<span class="o">=</span>0.5.0 -c conda-forge +conda install <span class="nv">pyarrow</span><span class="o">=</span>0.5.0 -c conda-forge </code></pre> </div> @@ -144,7 +148,7 @@ conda install pyarrow -c conda-forge <p>We have provided binary wheels on PyPI for Linux, macOS, and Windows:</p> -<div class="language-shell highlighter-rouge"><pre class="highlight"><code>pip install pyarrow +<div class="language-shell highlighter-rouge"><pre class="highlight"><code>pip install <span class="nv">pyarrow</span><span class="o">==</span>0.5.0 </code></pre> </div> http://git-wip-us.apache.org/repos/asf/arrow-site/blob/c4335214/release/0.5.0.html ---------------------------------------------------------------------- diff --git a/release/0.5.0.html b/release/0.5.0.html new file mode 100644 index 0000000..76ad517 --- /dev/null +++ b/release/0.5.0.html @@ -0,0 +1,300 @@ +<!DOCTYPE html> +<html lang="en-US"> + <head> + <meta charset="UTF-8"> + <title>Apache Arrow 0.5.0 Release</title> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <meta name="generator" content="Jekyll v3.4.3"> + <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> + <link rel="icon" type="image/x-icon" href="/favicon.ico"> + + <title>Apache Arrow Homepage</title> + <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900"> + + <link href="/css/main.css" rel="stylesheet"> + <link href="/css/syntax.css" rel="stylesheet"> + <script src="https://code.jquery.com/jquery-3.2.1.min.js"; + integrity="sha256-hwg4gsxgFZhOsEEamdOYGBf13FyQuiTwlAQgxVSNgt4=" + crossorigin="anonymous"></script> + <script src="/assets/javascripts/bootstrap.min.js"></script> + </head> + + +<body class="wrap"> + <div class="container"> + <nav class="navbar navbar-default"> + <div class="container-fluid"> + <div class="navbar-header"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#arrow-navbar"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="navbar-brand" href="/">Apache Arrow™ </a> + </div> + + <!-- Collect the nav links, forms, and other content for toggling --> + <div class="collapse navbar-collapse" id="arrow-navbar"> + <ul class="nav navbar-nav"> + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown" + role="button" aria-haspopup="true" + aria-expanded="false">Project Links<span class="caret"></span> + </a> + <ul class="dropdown-menu"> + <li><a href="/install/">Install</a></li> + <li><a href="/blog/">Blog</a></li> + <li><a href="/release/">Releases</a></li> + <li><a href="https://issues.apache.org/jira/browse/ARROW";>Issue Tracker</a></li> + <li><a href="https://github.com/apache/arrow";>Source Code</a></li> + <li><a href="http://mail-archives.apache.org/mod_mbox/arrow-dev/";>Mailing List</a></li> + <li><a href="https://apachearrowslackin.herokuapp.com";>Slack Channel</a></li> + <li><a href="/committers/">Committers</a></li> + </ul> + </li> + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown" + role="button" aria-haspopup="true" + aria-expanded="false">Specification<span class="caret"></span> + </a> + <ul class="dropdown-menu"> + <li><a href="/docs/memory_layout.html">Memory Layout</a></li> + <li><a href="/docs/metadata.html">Metadata</a></li> + <li><a href="/docs/ipc.html">Messaging / IPC</a></li> + </ul> + </li> + + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown" + role="button" aria-haspopup="true" + aria-expanded="false">Documentation<span class="caret"></span> + </a> + <ul class="dropdown-menu"> + <li><a href="/docs/python">Python</a></li> + <li><a href="/docs/cpp">C++ API</a></li> + <li><a href="/docs/java">Java API</a></li> + <li><a href="/docs/c_glib">C GLib API</a></li> + </ul> + </li> + <!-- <li><a href="/blog">Blog</a></li> --> + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown" + role="button" aria-haspopup="true" + aria-expanded="false">ASF Links<span class="caret"></span> + </a> + <ul class="dropdown-menu"> + <li><a href="http://www.apache.org/";>ASF Website</a></li> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Donate</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </li> + </ul> + <a href="http://www.apache.org/";> + <img style="float:right;" src="/img/asf_logo.svg" width="120px"/> + </a> + </div><!-- /.navbar-collapse --> + </div> + </nav> + + + <!-- + +--> + +<h1 id="apache-arrow-050-23-july-2017">Apache Arrow 0.5.0 (23 July 2017)</h1> + +<p>This is a major release, with expanded features in the supported languages and +additional integration test coverage between Java and C++.</p> + +<p>Read more in the <a href="http://arrow.apache.org/blog/2017/07/25/0.5.0-release/";>release blog post</a>.</p> + +<h2 id="download">Download</h2> + +<ul> + <li><a href="https://www.apache.org/dyn/closer.cgi/arrow/arrow-0.5.0/";><strong>Source Artifacts</strong></a></li> + <li><a href="https://github.com/apache/arrow/releases/tag/apache-arrow-0.5.0";>Git tag</a></li> +</ul> + +<h2 id="contributors">Contributors</h2> + +<div class="language-shell highlighter-rouge"><pre class="highlight"><code><span class="gp">$ </span>git shortlog -sn apache-arrow-0.4.1..apache-arrow-0.5.0 + 42 Wes McKinney + 22 Uwe L. Korn + 12 Kouhei Sutou + 9 Max Risuhin + 9 Phillip Cloud + 6 Philipp Moritz + 5 Steven Phillips + 3 Julien Le Dem + 2 Bryan Cutler + 2 Kengo Seki + 2 Max Risukhin + 2 fjetter + 1 Antony Mayi + 1 Deepak Majeti + 1 Fang Zheng + 1 Hideo Hattori + 1 Holden Karau + 1 Itai Incze + 1 Jeff Knupp + 1 LynnYuan + 1 Mark Lavrynenko + 1 Michael König + 1 Robert Nishihara + 1 Sudheesh Katkam + 1 Zahari + 1 vkorukanti +</code></pre> +</div> + +<h1 id="changelog">Changelog</h1> + +<h2 id="new-features-and-improvements">New Features and Improvements</h2> + +<ul> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1041";>ARROW-1041</a> - [Python] Support read_pandas on a directory of Parquet files</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1048";>ARROW-1048</a> - Allow user LD_LIBRARY_PATH to be used with source release script</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1052";>ARROW-1052</a> - Arrow 0.5.0 release</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1073";>ARROW-1073</a> - C++: Adapative integer builder</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1095";>ARROW-1095</a> - [Website] Add Arrow icon asset</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1100";>ARROW-1100</a> - [Python] Add âmodeâ property to NativeFile instances</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1102";>ARROW-1102</a> - Make MessageSerializer.serializeMessage() public</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-111";>ARROW-111</a> - [C++] Add static analyzer to tool chain to verify checking of Status returns</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1120";>ARROW-1120</a> - [Python] Write support for int96</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1122";>ARROW-1122</a> - [Website] Guest blog post on Arrow + ODBC from turbodbc</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1123";>ARROW-1123</a> - C++: Make jemalloc the default allocator</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1135";>ARROW-1135</a> - Upgrade Travis CI clang builds to use LLVM 4.0</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1137";>ARROW-1137</a> - Python: Ensure Pandas roundtrip of all-None column</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1142";>ARROW-1142</a> - [C++] Move over compression library toolchain from parquet-cpp</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1145";>ARROW-1145</a> - [GLib] Add get_values()</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1146";>ARROW-1146</a> - Add .gitignore for *_generated.h files in src/plasma/format</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1148";>ARROW-1148</a> - [C++] Raise minimum CMake version to 3.2</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1151";>ARROW-1151</a> - [C++] Add gcc branch prediction to status check macro</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1154";>ARROW-1154</a> - [C++] Migrate more computational utility code from parquet-cpp</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1160";>ARROW-1160</a> - C++: Implement DictionaryBuilder</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1165";>ARROW-1165</a> - [C++] Refactor PythonDecimalToArrowDecimal to not use templates</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1172";>ARROW-1172</a> - [C++] Use unique_ptr with array builder classes</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1183";>ARROW-1183</a> - [Python] Implement time type conversions in to_pandas</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1185";>ARROW-1185</a> - [C++] Clean up arrow::Status implementation, add warn_unused_result attribute for clang</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1187";>ARROW-1187</a> - Serialize a DataFrame with None column</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1193";>ARROW-1193</a> - [C++] Support pkg-config forarrow_python.so</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1196";>ARROW-1196</a> - [C++] Appveyor separate jobs for Debug/Release builds from sources; Build with conda toolchain; Build with NMake Makefiles Generator</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1198";>ARROW-1198</a> - Python: Add public C++ API to unwrap PyArrow object</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1199";>ARROW-1199</a> - [C++] Introduce mutable POD struct for generic array data</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1202";>ARROW-1202</a> - Remove semicolons from status macros</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1212";>ARROW-1212</a> - [GLib] Add garrow_binary_array_get_offsets_buffer()</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1214";>ARROW-1214</a> - [Python] Add classes / functions to enable stream message components to be handled outside of the stream reader class</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1217";>ARROW-1217</a> - [GLib] Add GInputStream based arrow::io::RandomAccessFile</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1220";>ARROW-1220</a> - [C++] Standartize usage of *_HOME cmake script variables for 3rd party libs</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1221";>ARROW-1221</a> - [C++] Pin clang-format version</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1227";>ARROW-1227</a> - [GLib] Support GOutputStream</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1228";>ARROW-1228</a> - [GLib] Test file name should be the same name as target class</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1229";>ARROW-1229</a> - [GLib] Follow Reader API change (get -> read)</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1233";>ARROW-1233</a> - [C++] Validate cmake script resolving of 3rd party linked libs from correct location in toolchain build</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-460";>ARROW-460</a> - [C++] Implement JSON round trip for DictionaryArray</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-462";>ARROW-462</a> - [C++] Implement in-memory conversions between non-nested primitive types and DictionaryArray equivalent</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-575";>ARROW-575</a> - Python: Auto-detect nested lists and nested numpy arrays in Pandas</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-597";>ARROW-597</a> - [Python] Add convenience function to yield DataFrame from any object that a StreamReader or FileReader can read from</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-599";>ARROW-599</a> - [C++] Add LZ4 codec to 3rd-party toolchain</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-600";>ARROW-600</a> - [C++] Add ZSTD codec to 3rd-party toolchain</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-692";>ARROW-692</a> - Java<->C++ Integration tests for dictionary-encoded vectors</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-693";>ARROW-693</a> - [Java] Add JSON support for dictionary vectors</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-742";>ARROW-742</a> - Handling exceptions during execution of std::wstring_convert</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-834";>ARROW-834</a> - [Python] Support creating Arrow arrays from Python iterables</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-915";>ARROW-915</a> - Struct Array reads limited support</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-935";>ARROW-935</a> - [Java] Build Javadoc in Travis CI</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-960";>ARROW-960</a> - [Python] Add source build guide for macOS + Homebrew</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-962";>ARROW-962</a> - [Python] Add schema attribute to FileReader</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-966";>ARROW-966</a> - [Python] pyarrow.list_ should also accept Field instance</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-978";>ARROW-978</a> - [Python] Use sphinx-bootstrap-theme for Sphinx documentation</li> +</ul> + +<h2 id="bug-fixes">Bug Fixes</h2> + +<ul> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1074";>ARROW-1074</a> - from_pandas doesnt convert ndarray to list</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1079";>ARROW-1079</a> - [Python] Empty âprivateâ directories should be ignored by Parquet interface</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1081";>ARROW-1081</a> - C++: arrow::test::TestBase::MakePrimitive doesnât fill null_bitmap</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1096";>ARROW-1096</a> - [C++] Memory mapping file over 4GB fails on Windows</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1097";>ARROW-1097</a> - Reading tensor needs file to be opened in writeable mode</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1098";>ARROW-1098</a> - Document Error?</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1101";>ARROW-1101</a> - UnionListWriter is not implementing all methods on interface ScalarWriter</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1103";>ARROW-1103</a> - [Python] Utilize pandas metadata from common _metadata Parquet file if it exists</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1107";>ARROW-1107</a> - [JAVA] NullableMapVector getField() should return nullable type</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1108";>ARROW-1108</a> - Check if ArrowBuf is empty buffer in getActualConsumedMemory() and getPossibleConsumedMemory()</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1109";>ARROW-1109</a> - [JAVA] transferOwnership fails when readerIndex is not 0</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1110";>ARROW-1110</a> - [JAVA] make union vector naming consistent</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1111";>ARROW-1111</a> - [JAVA] Make aligning buffers optional, and allow -1 for unknown null count</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1112";>ARROW-1112</a> - [JAVA] Set lastSet for VarLength and List vectors when loading</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1113";>ARROW-1113</a> - [C++] gflags EP build gets triggered (as a no-op) on subsequent calls to make or ninja build</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1115";>ARROW-1115</a> - [C++] Use absolute path for ccache</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1117";>ARROW-1117</a> - [Docs] Minor issues in GLib README</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1124";>ARROW-1124</a> - [Python] pyarrow needs to depend on numpy>=1.10 (not 1.9)</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1125";>ARROW-1125</a> - Python: Table.from_pandas doesnât work anymore on partial schemas</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1128";>ARROW-1128</a> - [Docs] command to build a wheel is not properly rendered</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1129";>ARROW-1129</a> - [C++] Fix Linux toolchain build regression from ARROW-742</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1131";>ARROW-1131</a> - Python: Parquet unit tests are always skipped</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1132";>ARROW-1132</a> - [Python] Unable to write pandas DataFrame w/MultiIndex containing duplicate values to parquet</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1136";>ARROW-1136</a> - [C++/Python] Segfault on empty stream</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1138";>ARROW-1138</a> - Travis: Use OpenJDK7 instead of OracleJDK7</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1139";>ARROW-1139</a> - [C++] dlmalloc doesnât allow arrow to be built with clang 4 or gcc 7.1.1</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1141";>ARROW-1141</a> - on import get libjemalloc.so.2: cannot allocate memory in static TLS block</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1143";>ARROW-1143</a> - C++: Fix comparison of NullArray</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1144";>ARROW-1144</a> - [C++] Remove unused variable</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1147";>ARROW-1147</a> - [C++] Allow optional vendoring of flatbuffers in plasma</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1150";>ARROW-1150</a> - [C++] AdaptiveIntBuilder compiler warning on MSVC</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1152";>ARROW-1152</a> - [Cython] read_tensor should work with a readable file</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1155";>ARROW-1155</a> - segmentation fault when run pa.Int16Value()</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1157";>ARROW-1157</a> - C++/Python: Decimal templates are not correctly exported on OSX</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1159";>ARROW-1159</a> - [C++] Static data members cannot be accessed from inline functions in Arrow headers by thirdparty users</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1162";>ARROW-1162</a> - Transfer Between Empty Lists Should Not Invoke Callback</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1166";>ARROW-1166</a> - Errors in Struct typeâs example and missing reference in Layout.md</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1167";>ARROW-1167</a> - [Python] Create chunked BinaryArray in Table.from_pandas when a columnâs data exceeds 2GB</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1168";>ARROW-1168</a> - [Python] pandas metadata may contain âmixedâ data types</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1169";>ARROW-1169</a> - C++: jemalloc externalproject doesnât build with CMakeâs ninja generator</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1170";>ARROW-1170</a> - C++: ARROW_JEMALLOC=OFF breaks linking on unittest</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1174";>ARROW-1174</a> - [GLib] Investigate root cause of ListArray glib test failure</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1177";>ARROW-1177</a> - [C++] Detect int32 overflow in ListBuilder::Append</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1179";>ARROW-1179</a> - C++: Add missing virtual destructors</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1180";>ARROW-1180</a> - [GLib] garrow_tensor_get_dimension_name() returns invalid address</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1181";>ARROW-1181</a> - [Python] Parquet test fail if not enabled</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1182";>ARROW-1182</a> - C++: Specify BUILD_BYPRODUCTS for zlib and zstd</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1186";>ARROW-1186</a> - [C++] Enable option to build arrow with minimal dependencies needed to build Parquet library</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1188";>ARROW-1188</a> - Segfault when trying to serialize a DataFrame with Null-only Categorical Column</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1190";>ARROW-1190</a> - VectorLoader corrupts vectors with duplicate names</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1191";>ARROW-1191</a> - [JAVA] Implement getField() method for the complex readers</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1194";>ARROW-1194</a> - Getting record batch size with pa.get_record_batch_size returns a size that is too small for pandas DataFrame.</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1197";>ARROW-1197</a> - [GLib] record_batch.hpp Inclusion is missing</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1200";>ARROW-1200</a> - [C++] DictionaryBuilder should use signed integers for indices</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1201";>ARROW-1201</a> - [Python] Incomplete Python types cause a core dump when repr-ing</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1203";>ARROW-1203</a> - [C++] Disallow BinaryBuilder to append byte strings larger than the maximum value of int32_t</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1205";>ARROW-1205</a> - C++: Reference to type objects in ArrayLoader may cause segmentation faults.</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1206";>ARROW-1206</a> - [C++] Enable MSVC builds to work with some compression library support disabled</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1208";>ARROW-1208</a> - [C++] Toolchain build with ZSTD library from conda-forge failure</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1215";>ARROW-1215</a> - [Python] Class methods in API reference</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1216";>ARROW-1216</a> - Numpy arrays cannot be created from Arrow Buffers on Python 2</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1218";>ARROW-1218</a> - Arrow doesnât compile if all compression libraries are deactivated</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1222";>ARROW-1222</a> - [Python] pyarrow.array returns NullArray for array of unsupported Python objects</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1223";>ARROW-1223</a> - [GLib] Fix function name that returns wrapped object</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1235";>ARROW-1235</a> - [C++] macOS linker failure with operator« and std::ostream</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-1236";>ARROW-1236</a> - Library paths in exported pkg-config file are incorrect</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-601";>ARROW-601</a> - Some logical types not supported when loading Parquet</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-784";>ARROW-784</a> - Cleaning up thirdparty toolchain support in Arrow on Windows</li> + <li><a href="https://issues.apache.org/jira/browse/ARROW-992";>ARROW-992</a> - [Python] In place development builds do not have a <strong>version</strong></li> +</ul> + + + + <hr/> +<footer class="footer"> + <p>Apache Arrow, Arrow, Apache, the Apache feather logo, and the Apache Arrow project logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.</p> + <p>© 2017 Apache Software Foundation</p> +</footer> + + </div> +</body> +</html> http://git-wip-us.apache.org/repos/asf/arrow-site/blob/c4335214/release/index.html ---------------------------------------------------------------------- diff --git a/release/index.html b/release/index.html index 03474f9..a21a610 100644 --- a/release/index.html +++ b/release/index.html @@ -110,6 +110,7 @@ <p>Navigate to the release page for downloads and the changelog.</p> <ul> + <li><a href="/release/0.5.0.html">0.5.0 (23 July 2017)</a></li> <li><a href="/release/0.4.1.html">0.4.1 (9 June 2017)</a></li> <li><a href="/release/0.4.0.html">0.4.0 (22 May 2017)</a></li> <li><a href="/release/0.3.0.html">0.3.0 (5 May 2017)</a></li>
