from:"shiro"

[arrow] branch master updated (d7dc021 -> 445d2b4)

2020-05-25 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from d7dc021  ARROW-8633: [C++] Add ValidateAscii function
 add 445d2b4  ARROW-8912: [Ruby] Keep reference of Arrow::Buffer's data for 
GC

No new revisions were added by this update.

Summary of changes:
 ruby/red-arrow/lib/arrow/{time32-array.rb => buffer.rb} | 12 ++--
 ruby/red-arrow/lib/arrow/loader.rb  |  1 +
 ruby/red-arrow/test/test-buffer.rb  | 11 +++
 3 files changed, 18 insertions(+), 6 deletions(-)
 copy ruby/red-arrow/lib/arrow/{time32-array.rb => buffer.rb} (83%)

[arrow] branch master updated: ARROW-8830: [GLib] Add support for Tell against not seekable GIO output stream

2020-05-17 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new e5a33f1  ARROW-8830: [GLib] Add support for Tell against not seekable 
GIO output stream
e5a33f1 is described below

commit e5a33f1220705aec6a224b55d2a6f47fbd957603
Author: Sutou Kouhei 
AuthorDate: Mon May 18 09:23:52 2020 +0900

ARROW-8830: [GLib] Add support for Tell against not seekable GIO output 
stream

Closes #7207 from kou/glib-gio-output-define-tell

Authored-by: Sutou Kouhei 
Signed-off-by: Yosuke Shiro 
---
 c_glib/arrow-glib/output-stream.cpp   | 17 -
 c_glib/test/test-gio-output-stream.rb | 16 
 2 files changed, 24 insertions(+), 9 deletions(-)

diff --git a/c_glib/arrow-glib/output-stream.cpp 
b/c_glib/arrow-glib/output-stream.cpp
index 1653332..2c3ccaf 100644
--- a/c_glib/arrow-glib/output-stream.cpp
+++ b/c_glib/arrow-glib/output-stream.cpp
@@ -358,7 +358,8 @@ namespace garrow {
   class GIOOutputStream : public arrow::io::OutputStream {
   public:
 GIOOutputStream(GOutputStream *output_stream) :
-  output_stream_(output_stream) {
+  output_stream_(output_stream),
+  position_(0) {
   g_object_ref(output_stream_);
 }
 
@@ -386,15 +387,11 @@ namespace garrow {
 }
 
 arrow::Result Tell() const override {
-  if (!G_IS_SEEKABLE(output_stream_)) {
-std::string message("[gio-output-stream][tell] "
-"not seekable output stream: <");
-message += G_OBJECT_CLASS_NAME(G_OBJECT_GET_CLASS(output_stream_));
-message += ">";
-return arrow::Status::NotImplemented(message);
+  if (G_IS_SEEKABLE(output_stream_)) {
+return g_seekable_tell(G_SEEKABLE(output_stream_));
+  } else {
+return position_;
   }
-
-  return g_seekable_tell(G_SEEKABLE(output_stream_));
 }
 
 arrow::Status Write(const void *data,
@@ -408,6 +405,7 @@ namespace garrow {
  NULL,
  );
   if (successed) {
+position_ += n_written_bytes;
 return arrow::Status::OK();
   } else {
 std::stringstream message("[gio-output-stream][write]");
@@ -432,6 +430,7 @@ namespace garrow {
 
   private:
 GOutputStream *output_stream_;
+int64_t position_;
   };
 };
 
diff --git a/c_glib/test/test-gio-output-stream.rb 
b/c_glib/test/test-gio-output-stream.rb
index c77598e..36756cb 100644
--- a/c_glib/test/test-gio-output-stream.rb
+++ b/c_glib/test/test-gio-output-stream.rb
@@ -60,4 +60,20 @@ class TestGIOOutputStream < Test::Unit::TestCase
 output = Arrow::GIOOutputStream.new(output_stream)
 assert_equal(output_stream, output.raw)
   end
+
+  def test_tell
+unless Gio.const_defined?(:UnixOutputStream)
+  omit("Need Gio::UnixOutputStream")
+end
+tempfile = Tempfile.open("arrow-gio-output-stream")
+begin
+  output_stream = Gio::UnixOutputStream.new(tempfile.to_i, false)
+  output = Arrow::GIOOutputStream.new(output_stream)
+  assert_equal(0, output.tell)
+  output.write("Hello")
+  assert_equal(5, output.tell)
+ensure
+  tempfile.close!
+end
+  end
 end

[arrow] branch master updated: ARROW-8682: [Ruby][Parquet] Add support for column level compression

2020-05-04 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 0ba03b7  ARROW-8682: [Ruby][Parquet] Add support for column level 
compression
0ba03b7 is described below

commit 0ba03b73530c10162f4db667c71ca2be0b71a1dd
Author: Sutou Kouhei 
AuthorDate: Mon May 4 21:50:51 2020 +0900

ARROW-8682: [Ruby][Parquet] Add support for column level compression

Closes #7092 from kou/ruby-parquet-compression

Authored-by: Sutou Kouhei 
Signed-off-by: Yosuke Shiro 
---
 ruby/red-arrow/lib/arrow/table-saver.rb| 12 ++---
 ruby/red-arrow/test/test-feather.rb|  2 +-
 .../red-parquet/lib/parquet/arrow-table-savable.rb | 20 +++-
 ruby/red-parquet/lib/parquet/loader.rb |  1 +
 ...arrow-table-savable.rb => writer-properties.rb} | 20 +++-
 ruby/red-parquet/test/test-arrow-table.rb  | 53 --
 6 files changed, 80 insertions(+), 28 deletions(-)

diff --git a/ruby/red-arrow/lib/arrow/table-saver.rb 
b/ruby/red-arrow/lib/arrow/table-saver.rb
index 6c44b99..bc2296a 100644
--- a/ruby/red-arrow/lib/arrow/table-saver.rb
+++ b/ruby/red-arrow/lib/arrow/table-saver.rb
@@ -155,13 +155,13 @@ module Arrow
 end
 
 def save_as_feather
+  properties = FeatherWriteProperties.new
+  properties.class.properties.each do |name|
+value = @options[name.to_sym]
+next if value.nil?
+properties.__send__("#{name}=", value)
+  end
   open_raw_output_stream do |output|
-properties = FeatherWriteProperties.new
-properties.class.properties.each do |name|
-  value = @options[name.to_sym]
-  next if value.nil?
-  properties.__send__("#{name}=", value)
-end
 @table.write_as_feather(output, properties)
   end
 end
diff --git a/ruby/red-arrow/test/test-feather.rb 
b/ruby/red-arrow/test/test-feather.rb
index d36df5e..21d8a2c 100644
--- a/ruby/red-arrow/test/test-feather.rb
+++ b/ruby/red-arrow/test/test-feather.rb
@@ -28,7 +28,7 @@ class FeatherTest < Test::Unit::TestCase
 @output = Tempfile.new(["red-arrow", ".feather"])
 begin
   yield(@output)
-rescue
+ensure
   @output.close!
 end
   end
diff --git a/ruby/red-parquet/lib/parquet/arrow-table-savable.rb 
b/ruby/red-parquet/lib/parquet/arrow-table-savable.rb
index 7667381..0163b15 100644
--- a/ruby/red-parquet/lib/parquet/arrow-table-savable.rb
+++ b/ruby/red-parquet/lib/parquet/arrow-table-savable.rb
@@ -19,9 +19,25 @@ module Parquet
   module ArrowTableSavable
 private
 def save_as_parquet
+  properties = WriterProperties.new
+  @options.each do |key, value|
+next if value.nil?
+set_method_name = "set_#{key}"
+next unless properties.respond_to?(set_method_name)
+case value
+when ::Array, ::Hash
+  value.each do |path, v|
+properties.__send__(set_method_name, v, path)
+  end
+else
+  properties.__send__(set_method_name, value)
+end
+  end
   chunk_size = @options[:chunk_size] || 1024 # TODO
-  open_output_stream do |output|
-Parquet::ArrowFileWriter.open(@table.schema, output) do |writer|
+  open_raw_output_stream do |output|
+ArrowFileWriter.open(@table.schema,
+ output,
+ properties) do |writer|
   writer.write_table(@table, chunk_size)
 end
   end
diff --git a/ruby/red-parquet/lib/parquet/loader.rb 
b/ruby/red-parquet/lib/parquet/loader.rb
index a3d0cb4..5e25872 100644
--- a/ruby/red-parquet/lib/parquet/loader.rb
+++ b/ruby/red-parquet/lib/parquet/loader.rb
@@ -31,6 +31,7 @@ module Parquet
 def require_libraries
   require "parquet/arrow-table-loadable"
   require "parquet/arrow-table-savable"
+  require "parquet/writer-properties"
 end
 
 def load_object_info(info)
diff --git a/ruby/red-parquet/lib/parquet/arrow-table-savable.rb 
b/ruby/red-parquet/lib/parquet/writer-properties.rb
similarity index 69%
copy from ruby/red-parquet/lib/parquet/arrow-table-savable.rb
copy to ruby/red-parquet/lib/parquet/writer-properties.rb
index 7667381..5881471 100644
--- a/ruby/red-parquet/lib/parquet/arrow-table-savable.rb
+++ b/ruby/red-parquet/lib/parquet/writer-properties.rb
@@ -16,21 +16,13 @@
 # under the License.
 
 module Parquet
-  module ArrowTableSavable
-private
-def save_as_parquet
-  chunk_size = @options[:chunk_size] || 1024 # TODO
-  open_output_stream do |output|
-Parquet::ArrowFileWriter.open(@table.schema, output) do |writer|
-  writer.write_table(@table, chunk_size)
-end
+  class WriterProperties
+def set_dict

[arrow] branch master updated: ARROW-8509: [GLib] Add low level record batch read/write functions

2020-05-03 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new efc7b81  ARROW-8509: [GLib] Add low level record batch read/write 
functions
efc7b81 is described below

commit efc7b816177ba0bc3b8fed86115cae2e538c5cf3
Author: Sutou Kouhei 
AuthorDate: Sun May 3 23:02:13 2020 +0900

ARROW-8509: [GLib] Add low level record batch read/write functions

Closes #7086 from kou/glib-record-batch-serialize

Authored-by: Sutou Kouhei 
Signed-off-by: Yosuke Shiro 
---
 c_glib/arrow-glib/Makefile.am|  1 +
 c_glib/arrow-glib/arrow-glib.hpp |  1 +
 c_glib/arrow-glib/input-stream.cpp   | 57 ++
 c_glib/arrow-glib/input-stream.h |  8 +
 c_glib/arrow-glib/ipc-options.cpp| 28 +--
 c_glib/arrow-glib/ipc-options.hpp| 32 +
 c_glib/arrow-glib/meson.build|  1 +
 c_glib/arrow-glib/output-stream.cpp  | 59 +---
 c_glib/arrow-glib/output-stream.h|  8 +
 c_glib/arrow-glib/record-batch.cpp   | 59 
 c_glib/arrow-glib/record-batch.h |  8 -
 c_glib/test/test-buffer-input-stream.rb  | 25 ++
 c_glib/test/test-buffer-output-stream.rb | 25 ++
 c_glib/test/test-record-batch.rb |  7 
 14 files changed, 304 insertions(+), 15 deletions(-)

diff --git a/c_glib/arrow-glib/Makefile.am b/c_glib/arrow-glib/Makefile.am
index 2c07501..8e02ce3 100644
--- a/c_glib/arrow-glib/Makefile.am
+++ b/c_glib/arrow-glib/Makefile.am
@@ -182,6 +182,7 @@ libarrow_glib_la_cpp_headers += \
writable-file.hpp
 
 libarrow_glib_la_cpp_headers +=\
+   ipc-options.hpp \
metadata-version.hpp\
reader.hpp  \
writer.hpp
diff --git a/c_glib/arrow-glib/arrow-glib.hpp b/c_glib/arrow-glib/arrow-glib.hpp
index d755b2b..1d58b48 100644
--- a/c_glib/arrow-glib/arrow-glib.hpp
+++ b/c_glib/arrow-glib/arrow-glib.hpp
@@ -44,6 +44,7 @@
 #include 
 #include 
 
+#include 
 #include 
 #include 
 #include 
diff --git a/c_glib/arrow-glib/input-stream.cpp 
b/c_glib/arrow-glib/input-stream.cpp
index c98442e..7b8419b 100644
--- a/c_glib/arrow-glib/input-stream.cpp
+++ b/c_glib/arrow-glib/input-stream.cpp
@@ -31,7 +31,10 @@
 #include 
 #include 
 #include 
+#include 
 #include 
+#include 
+#include 
 #include 
 
 #include 
@@ -300,6 +303,60 @@ garrow_input_stream_read_tensor(GArrowInputStream 
*input_stream,
   }
 }
 
+/**
+ * garrow_input_stream_read_record_batch:
+ * @input_stream: A #GArrowInputStream.
+ * @schema: A #GArrowSchema for a read record batch.
+ * @options: (nullable): A #GArrowReadOptions.
+ * @error: (nullable): Return location for a #GError or %NULL.
+ *
+ * Returns: (transfer full) (nullable):
+ *   #GArrowRecordBatch on success, %NULL on error.
+ *
+ * Since: 1.0.0
+ */
+GArrowRecordBatch *
+garrow_input_stream_read_record_batch(GArrowInputStream *input_stream,
+  GArrowSchema *schema,
+  GArrowReadOptions *options,
+  GError **error)
+{
+  auto arrow_input_stream = garrow_input_stream_get_raw(input_stream);
+  auto arrow_schema = garrow_schema_get_raw(schema);
+
+  if (options) {
+auto arrow_options = garrow_read_options_get_raw(options);
+auto arrow_dictionary_memo =
+  garrow_read_options_get_dictionary_memo_raw(options);
+auto arrow_record_batch =
+  arrow::ipc::ReadRecordBatch(arrow_schema,
+  arrow_dictionary_memo,
+  *arrow_options,
+  arrow_input_stream.get());
+if (garrow::check(error,
+  arrow_record_batch,
+  "[input-stream][read-record-batch]")) {
+  return garrow_record_batch_new_raw(&(*arrow_record_batch));
+} else {
+  return NULL;
+}
+  } else {
+auto arrow_options = arrow::ipc::IpcReadOptions::Defaults();
+auto arrow_record_batch =
+  arrow::ipc::ReadRecordBatch(arrow_schema,
+  nullptr,
+  arrow_options,
+  arrow_input_stream.get());
+if (garrow::check(error,
+  arrow_record_batch,
+  "[input-stream][read-record-batch]")) {
+  return garrow_record_batch_new_raw(&(*arrow_record_batch));
+} else {
+  return NULL;
+}
+  }
+}
+
 
 G_DEFINE_TYPE(GArrowSeekableInputStream,
   garrow_seekable_input_stream,
diff --git a/c_glib/arrow-glib/input-stream.h b/c_glib/arrow-glib/input-stream.h
i

[arrow] branch master updated (52fb3c6 -> f60c0b1)

2020-03-12 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 52fb3c6  ARROW-7675: [R][CI] Move Windows CI from Appveyor to GHA
 add f60c0b1  ARROW-8083: [GLib] Add support for Peek() to GIOInputStream

No new revisions were added by this update.

Summary of changes:
 c_glib/arrow-glib/input-stream.cpp   | 31 +++
 c_glib/test/test-gio-input-stream.rb |  9 +
 2 files changed, 40 insertions(+)

[arrow] branch master updated: ARROW-7959: [Ruby] Add support for Ruby 2.3 again

2020-02-28 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 5e3255b  ARROW-7959: [Ruby] Add support for Ruby 2.3 again
5e3255b is described below

commit 5e3255b1499c3d504139998365e6cc3e57f118d6
Author: Sutou Kouhei 
AuthorDate: Fri Feb 28 20:48:36 2020 +0900

ARROW-7959: [Ruby] Add support for Ruby 2.3 again

Ruby 2.3 reached EOL but Ubuntu 16.04 LTS ships Ruby 2.3. So
supporting Ruby 2.3 again is valuable.

Note that Red Arrow 0.15.1 works with Ruby 2.3.

Closes #6501 from kou/ruby-add-support-for-ruby-2.3 and squashes the 
following commits:

f32018120   Add support for Ruby 2.3 again

Authored-by: Sutou Kouhei 
Signed-off-by: Yosuke Shiro 
---
 .github/workflows/cpp.yml  | 2 +-
 .github/workflows/ruby.yml | 5 -
 c_glib/test/test-numeric-array.rb  | 2 +-
 ruby/red-arrow/lib/arrow/generic-filterable.rb | 4 ++--
 ruby/red-arrow/lib/arrow/generic-takeable.rb   | 4 ++--
 ruby/red-arrow/lib/arrow/null-array-builder.rb | 2 +-
 6 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/.github/workflows/cpp.yml b/.github/workflows/cpp.yml
index 63ba695..ae694c1 100644
--- a/.github/workflows/cpp.yml
+++ b/.github/workflows/cpp.yml
@@ -93,7 +93,7 @@ jobs:
 run: |
   docker login -u ${{ secrets.DOCKERHUB_USER }} \
-p ${{ secrets.DOCKERHUB_TOKEN }}
-  docker-compose push ubuntu-cpp
+  docker-compose push ubuntu-cpp-sanitizer
 
   macos:
 name: AMD64 MacOS 10.15 C++
diff --git a/.github/workflows/ruby.yml b/.github/workflows/ruby.yml
index 0282f49..d5e3e59 100644
--- a/.github/workflows/ruby.yml
+++ b/.github/workflows/ruby.yml
@@ -41,7 +41,9 @@ jobs:
 strategy:
   fail-fast: false
   matrix:
-ubuntu: [18.04]
+ubuntu:
+  - 16.04
+  - 18.04
 env:
   UBUNTU: ${{ matrix.ubuntu }}
 steps:
@@ -71,6 +73,7 @@ jobs:
 run: |
   docker login -u ${{ secrets.DOCKERHUB_USER }} \
-p ${{ secrets.DOCKERHUB_TOKEN }}
+  docker-compose push ubuntu-c-glib
   docker-compose push ubuntu-ruby
 
   macos:
diff --git a/c_glib/test/test-numeric-array.rb 
b/c_glib/test/test-numeric-array.rb
index d919d59..f90007c 100644
--- a/c_glib/test/test-numeric-array.rb
+++ b/c_glib/test/test-numeric-array.rb
@@ -20,7 +20,7 @@ class TestNumericArray < Test::Unit::TestCase
 
   def test_mean
 array = build_double_array([1.1, 2.2, nil])
-assert_in_delta(array.values.sum / 2,
+assert_in_delta(array.values.inject(&:+) / 2,
 array.mean)
   end
 end
diff --git a/ruby/red-arrow/lib/arrow/generic-filterable.rb 
b/ruby/red-arrow/lib/arrow/generic-filterable.rb
index 9e0ba55..4fd5c87 100644
--- a/ruby/red-arrow/lib/arrow/generic-filterable.rb
+++ b/ruby/red-arrow/lib/arrow/generic-filterable.rb
@@ -19,8 +19,8 @@ module Arrow
   module GenericFilterable
 class << self
   def included(base)
-base.alias_method :filter_raw, :filter
-base.alias_method :filter, :filter_generic
+base.__send__(:alias_method, :filter_raw, :filter)
+base.__send__(:alias_method, :filter, :filter_generic)
   end
 end
 
diff --git a/ruby/red-arrow/lib/arrow/generic-takeable.rb 
b/ruby/red-arrow/lib/arrow/generic-takeable.rb
index 3aa63dde..f32b43f 100644
--- a/ruby/red-arrow/lib/arrow/generic-takeable.rb
+++ b/ruby/red-arrow/lib/arrow/generic-takeable.rb
@@ -19,8 +19,8 @@ module Arrow
   module GenericTakeable
 class << self
   def included(base)
-base.alias_method :take_raw, :take
-base.alias_method :take, :take_generic
+base.__send__(:alias_method, :take_raw, :take)
+base.__send__(:alias_method, :take, :take_generic)
   end
 end
 
diff --git a/ruby/red-arrow/lib/arrow/null-array-builder.rb 
b/ruby/red-arrow/lib/arrow/null-array-builder.rb
index 01b7604..26e58cc 100644
--- a/ruby/red-arrow/lib/arrow/null-array-builder.rb
+++ b/ruby/red-arrow/lib/arrow/null-array-builder.rb
@@ -19,7 +19,7 @@ module Arrow
   class NullArrayBuilder
 class << self
   def buildable?(args)
-super and args.collect(&:class) != [Integer]
+super and not (args.size == 1 and args[0].is_a?(Integer))
   end
 end
   end

[arrow] branch master updated (b557587 -> ccbdf3c)

2020-02-26 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from b557587  ARROW-7877: [Packaging] Fix crossbow deployment to github 
artifacts
 add ccbdf3c  ARROW-7625: [Parquet][GLib] Add support for writer properties

No new revisions were added by this update.

Summary of changes:
 .github/workflows/ruby.yml|   1 +
 c_glib/parquet-glib/arrow-file-writer.cpp | 343 +-
 c_glib/parquet-glib/arrow-file-writer.h   |  69 ++
 c_glib/parquet-glib/arrow-file-writer.hpp |   4 +
 c_glib/parquet-glib/parquet-glib.h|   2 +
 c_glib/parquet-glib/version.h.in  |  23 ++
 c_glib/test/parquet/test-writer-properties.rb | 103 
 7 files changed, 533 insertions(+), 12 deletions(-)
 create mode 100644 c_glib/test/parquet/test-writer-properties.rb

[arrow] branch master updated (17220bb -> d3f4763)

2020-01-21 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 17220bb  ARROW-7589: [C++][Gandiva] Calling castVarchar from java 
sometimes results in segmentation fault for input length 0
 add d3f4763  ARROW-7626: [Parquet][GLib] Add support for version macros

No new revisions were added by this update.

Summary of changes:
 c_glib/.gitignore |   1 +
 c_glib/configure.ac   |  10 ++
 c_glib/doc/parquet-glib/parquet-glib-docs.xml |   8 ++
 c_glib/parquet-glib/Makefile.am   |   9 +-
 c_glib/parquet-glib/meson.build   |  10 ++
 c_glib/parquet-glib/version.h.in  | 181 ++
 6 files changed, 218 insertions(+), 1 deletion(-)
 create mode 100644 c_glib/parquet-glib/version.h.in

[arrow] branch master updated: ARROW-7474: [Ruby] Improve CSV save performance

2019-12-30 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 8d94859  ARROW-7474: [Ruby] Improve CSV save performance
8d94859 is described below

commit 8d94859161c99cfc4a4b0fb3a513bb89484cb4eb
Author: Sutou Kouhei 
AuthorDate: Tue Dec 31 10:03:25 2019 +0900

ARROW-7474: [Ruby] Improve CSV save performance

Closes #6106 from kou/ruby-csv-save-improve and squashes the following 
commits:

95c1e99dc  ARROW-7474:  Improve CSV save performance

Authored-by: Sutou Kouhei 
Signed-off-by: Yosuke Shiro 
---
 ruby/red-arrow/lib/arrow/table-saver.rb | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/ruby/red-arrow/lib/arrow/table-saver.rb 
b/ruby/red-arrow/lib/arrow/table-saver.rb
index 9ffa843..f098ab6 100644
--- a/ruby/red-arrow/lib/arrow/table-saver.rb
+++ b/ruby/red-arrow/lib/arrow/table-saver.rb
@@ -140,10 +140,8 @@ module Arrow
 csv = CSV.new(output, **options)
 names = @table.schema.fields.collect(&:name)
 csv << names
-@table.each_record(reuse_record: true) do |record|
-  csv << names.collect do |name|
-record[name]
-  end
+@table.raw_records.each do |record|
+  csv << record
 end
   end
 end

[arrow] branch master updated: ARROW-7276: [Ruby] Add support for building Arrow::ListArray from [[...]]

2019-11-29 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new be6fad5  ARROW-7276: [Ruby] Add support for building Arrow::ListArray 
from [[...]]
be6fad5 is described below

commit be6fad5f489a74d2984f139a0ae55f45f4f3e23d
Author: Sutou Kouhei 
AuthorDate: Sat Nov 30 08:58:48 2019 +0900

ARROW-7276: [Ruby] Add support for building Arrow::ListArray from [[...]]

Closes #5925 from kou/ruby-array-build-list and squashes the following 
commits:

efad24b1e   Add support for building Arrow::ListArray from ]

Authored-by: Sutou Kouhei 
Signed-off-by: Yosuke Shiro 
---
 ruby/red-arrow/lib/arrow/array-builder.rb | 153 --
 ruby/red-arrow/test/test-array-builder.rb |  17 
 2 files changed, 118 insertions(+), 52 deletions(-)

diff --git a/ruby/red-arrow/lib/arrow/array-builder.rb 
b/ruby/red-arrow/lib/arrow/array-builder.rb
index ec7ae2c..9d6d594 100644
--- a/ruby/red-arrow/lib/arrow/array-builder.rb
+++ b/ruby/red-arrow/lib/arrow/array-builder.rb
@@ -26,60 +26,13 @@ module Arrow
   return builder.build(values)
 end
 
-builder_class = nil
-builder_class_arguments = []
+builder_info = nil
 values.each do |value|
-  case value
-  when nil
-# Ignore
-  when true, false
-return BooleanArray.new(values)
-  when String
-return StringArray.new(values)
-  when Float
-return DoubleArray.new(values)
-  when Integer
-if value < 0
-  builder = IntArrayBuilder.new
-  return builder.build(values)
-else
-  builder_class = UIntArrayBuilder
-  builder_class_arguments = []
-end
-  when Time
-data_type = value.data_type
-case data_type.unit
-when TimeUnit::SECOND
-  if builder.nil?
-builder = Time32ArrayBuilder
-builder_class_arguments = [data_type]
-  end
-when TimeUnit::MILLI
-  if builder != Time64ArrayBuilder
-builder = Time32ArrayBuilder
-builder_class_arguments = [data_type]
-  end
-when TimeUnit::MICRO
-  builder = Time64ArrayBuilder
-  builder_class_arguments = [data_type]
-when TimeUnit::NANO
-  builder = Time64ArrayBuilder.new(data_type)
-  return builder.build(values)
-end
-  when ::Time
-data_type = TimestampDataType.new(:nano)
-builder = TimestampArrayBuilder.new(data_type)
-return builder.build(values)
-  when DateTime
-return Date64Array.new(values)
-  when Date
-return Date32Array.new(values)
-  else
-return StringArray.new(values)
-  end
+  builder_info = detect_builder_info(value, builder_info)
+  break if builder_info and builder_info[:detected]
 end
-if builder_class
-  builder = builder_class.new(*builder_class_arguments)
+if builder_info
+  builder = builder_info[:builder]
   builder.build(values)
 else
   Arrow::StringArray.new(values)
@@ -89,6 +42,102 @@ module Arrow
   def buildable?(args)
 args.size == method(:build).arity
   end
+
+  private
+  def detect_builder_info(value, builder_info)
+case value
+when nil
+  builder_info
+when true, false
+  {
+builder: BooleanArrayBuilder.new,
+detected: true,
+  }
+when String
+  {
+builder: StringArrayBuilder.new,
+detected: true,
+  }
+when Float
+  {
+builder: DoubleArrayBuilder.new,
+detected: true,
+  }
+when Integer
+  if value < 0
+{
+  builder: IntArrayBuilder.new,
+  detected: true,
+}
+  else
+{
+  builder: UIntArrayBuilder.new,
+}
+  end
+when Time
+  data_type = value.data_type
+  case data_type.unit
+  when TimeUnit::SECOND
+builder_info || {
+  builder: Time32ArrayBuilder.new(data_type)
+}
+  when TimeUnit::MILLI
+if builder_info and 
builder_info[:builder].is_a?(Time64ArrayBuilder)
+  builder_info
+else
+  {
+builder: Time32ArrayBuilder.new(data_type),
+  }
+end
+  when TimeUnit::MICRO
+{
+  builder: Time64ArrayBuilder.new(dat

[arrow] branch master updated: ARROW-7275: [Ruby] Add support for Arrow::ListDataType.new(data_type)

2019-11-29 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 03874ef  ARROW-7275: [Ruby] Add support for 
Arrow::ListDataType.new(data_type)
03874ef is described below

commit 03874ef5c66f49370218ae60d048e7bf33561c6a
Author: Sutou Kouhei 
AuthorDate: Sat Nov 30 08:33:25 2019 +0900

ARROW-7275: [Ruby] Add support for Arrow::ListDataType.new(data_type)

"item" is used as the default field name for this case.

Closes #5924 from kou/ruby-list-data-type-new and squashes the following 
commits:

0a4cfb7a8   Add support for Arrow::ListDataType.new(data_type)

Authored-by: Sutou Kouhei 
Signed-off-by: Yosuke Shiro 
---
 ruby/red-arrow/lib/arrow/list-data-type.rb | 66 ++
 ruby/red-arrow/test/test-list-data-type.rb | 28 -
 2 files changed, 85 insertions(+), 9 deletions(-)

diff --git a/ruby/red-arrow/lib/arrow/list-data-type.rb 
b/ruby/red-arrow/lib/arrow/list-data-type.rb
index c097da4..cfcdd2a 100644
--- a/ruby/red-arrow/lib/arrow/list-data-type.rb
+++ b/ruby/red-arrow/lib/arrow/list-data-type.rb
@@ -53,16 +53,66 @@ module Arrow
 #
 #   @example Create a list data type with field description
 # Arrow::ListDataType.new(field: {name: "visible", type: :boolean})
-def initialize(field)
-  if field.is_a?(Hash) and field.key?(:field)
-description = field
-field = description[:field]
-  end
-  if field.is_a?(Hash)
-field_description = field
-field = Field.new(field_description)
+#
+# @overload initialize(data_type)
+#
+#   @param data_type [Arrow::DataType, String, Symbol,
+# ::Array, ::Array, Hash] The element data
+# type of the list data type. A field is created with the
+# default name `"item"` from the data type automatically.
+#
+# See {Arrow::DataType.resolve} how to specify data type.
+#
+#   @example Create a list data type with {Arrow::DataType}
+# Arrow::ListDataType.new(Arrow::BooleanDataType.new)
+#
+#   @example Create a list data type with data type name as String
+# Arrow::ListDataType.new("boolean")
+#
+#   @example Create a list data type with data type name as Symbol
+# Arrow::ListDataType.new(:boolean)
+#
+#   @example Create a list data type with data type as Array
+# Arrow::ListDataType.new([:time32, :milli])
+def initialize(arg)
+  data_type = resolve_data_type(arg)
+  if data_type
+field = Field.new(default_field_name, data_type)
+  else
+field = resolve_field(arg)
   end
   initialize_raw(field)
 end
+
+private
+def resolve_data_type(arg)
+  case arg
+  when DataType, String, Symbol, ::Array
+DataType.resolve(arg)
+  when Hash
+return nil if arg[:name]
+return nil unless arg[:type]
+DataType.resolve(arg)
+  else
+nil
+  end
+end
+
+def default_field_name
+  "item"
+end
+
+def resolve_field(arg)
+  if arg.is_a?(Hash) and arg.key?(:field)
+description = arg
+arg = description[:field]
+  end
+  if arg.is_a?(Hash)
+field_description = arg
+Field.new(field_description)
+  else
+arg
+  end
+end
   end
 end
diff --git a/ruby/red-arrow/test/test-list-data-type.rb 
b/ruby/red-arrow/test/test-list-data-type.rb
index cca6ca3..ada4639 100644
--- a/ruby/red-arrow/test/test-list-data-type.rb
+++ b/ruby/red-arrow/test/test-list-data-type.rb
@@ -23,7 +23,7 @@ class ListDataTypeTest < Test::Unit::TestCase
Arrow::ListDataType.new(field).to_s)
 end
 
-test("Hash") do
+test("name: String") do
   assert_equal("list",
Arrow::ListDataType.new(name: "tag", type: :string).to_s)
 end
@@ -39,5 +39,31 @@ class ListDataTypeTest < Test::Unit::TestCase
   assert_equal("list",
Arrow::ListDataType.new(field: field_description).to_s)
 end
+
+test("Arrow::DataType") do
+  data_type = Arrow::BooleanDataType.new
+  assert_equal("list",
+   Arrow::ListDataType.new(data_type).to_s)
+end
+
+test("String") do
+  assert_equal("list",
+   Arrow::ListDataType.new("boolean").to_s)
+end
+
+test("Symbol") do
+  assert_equal("list",
+   Arrow::ListDataType.new(:boolean).to_s)
+end
+
+test("[data type name, additional information]") do
+  assert_equal("list",
+   Arrow::ListDataType.new([:time32, :milli]).to_s)
+end
+
+test("type: Symbol") do
+  assert_equal("list",
+   Arrow::ListDataType.new(type: :boolean).to_s)
+end
   end
 end

[arrow] branch master updated (bb1e464 -> a313fea)

2019-11-18 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from bb1e464  ARROW-7178: [C++] Vendor forward compatible std::optional
 add a313fea  ARROW-7195: [Ruby] Improve #filter, #take, and #is_in

No new revisions were added by this update.

Summary of changes:
 ruby/red-arrow/lib/arrow/array.rb  | 38 ++---
 ruby/red-arrow/lib/arrow/chunked-array.rb  |  2 +
 .../lib/arrow/decimal128-array-builder.rb  |  2 -
 .../{compression-type.rb => generic-filterable.rb} | 30 ---
 .../{compression-type.rb => generic-takeable.rb}   | 25 +++---
 ruby/red-arrow/lib/arrow/loader.rb |  8 ++
 ruby/red-arrow/lib/arrow/record-batch.rb   |  3 -
 ruby/red-arrow/lib/arrow/schema.rb |  2 -
 ruby/red-arrow/lib/arrow/struct-data-type.rb   |  2 -
 ruby/red-arrow/lib/arrow/table.rb  | 18 +---
 ruby/red-arrow/test/test-array.rb  | 95 ++
 ruby/red-arrow/test/test-chunked-array.rb  | 94 +
 ruby/red-arrow/test/test-table.rb  | 36 
 ruby/red-arrow/test/test-timestamp-array.rb| 19 +
 14 files changed, 315 insertions(+), 59 deletions(-)
 copy ruby/red-arrow/lib/arrow/{compression-type.rb => generic-filterable.rb} 
(63%)
 copy ruby/red-arrow/lib/arrow/{compression-type.rb => generic-takeable.rb} 
(71%)

[arrow] branch master updated (99aa62b -> 1cd12ab)

2019-10-18 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 99aa62b  ARROW-6869: [C++] Do not return invalid arrays from 
DictionaryBuilder::Finish when reusing builder. Add "FinishDelta" method and 
"ResetFull" method
 add 1cd12ab  ARROW-6285: [GLib] Add support for LargeBinary and 
LargeString types

No new revisions were added by this update.

Summary of changes:
 c_glib/arrow-glib/array-builder.cpp| 508 -
 c_glib/arrow-glib/array-builder.h  |  98 
 c_glib/arrow-glib/basic-array.cpp  | 244 +-
 c_glib/arrow-glib/basic-array.h|  51 +++
 c_glib/arrow-glib/basic-data-type.cpp  |  79 
 c_glib/arrow-glib/basic-data-type.h|  34 +-
 c_glib/arrow-glib/type.cpp |   4 +
 c_glib/arrow-glib/type.h   |  15 +-
 c_glib/test/helper/buildable.rb|  10 +
 c_glib/test/run-test.sh|   8 +-
 c_glib/test/test-array-builder.rb  | 213 +
 ...-binary-array.rb => test-large-binary-array.rb} |  26 +-
 ...data-type.rb => test-large-binary-data-type.rb} |  10 +-
 ...-string-array.rb => test-large-string-array.rb} |  28 +-
 ...data-type.rb => test-large-string-data-type.rb} |  10 +-
 c_glib/test/test-string-array.rb   |   6 +-
 16 files changed, 1267 insertions(+), 77 deletions(-)
 copy c_glib/test/{test-binary-array.rb => test-large-binary-array.rb} (68%)
 copy c_glib/test/{test-date32-data-type.rb => test-large-binary-data-type.rb} 
(76%)
 copy c_glib/test/{test-string-array.rb => test-large-string-array.rb} (58%)
 copy c_glib/test/{test-date32-data-type.rb => test-large-string-data-type.rb} 
(76%)

[arrow] branch master updated (3bf4d80 -> dcc1e9d)

2019-09-16 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 3bf4d80  ARROW-6572: [C++] Fix Parquet decoding returning 
uninitialized data
 add dcc1e9d  ARROW-6562: [GLib] Fix returning wrong sliced data of 
GArrowBuffer

No new revisions were added by this update.

Summary of changes:
 c_glib/arrow-glib/buffer.cpp | 69 ++--
 c_glib/arrow-glib/buffer.hpp | 22 +-
 c_glib/test/test-buffer.rb   | 27 +
 3 files changed, 84 insertions(+), 34 deletions(-)

[arrow] branch master updated (d4d4a12 -> e29732b)

2019-08-28 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from d4d4a12  ARROW-6334: [Java] Improve the dictionary builder API to 
return the position of the value in the dictionary
 add e29732b  ARROW-6351: [Ruby] Improve Arrow#values performance

No new revisions were added by this update.

Summary of changes:
 .../red-arrow/benchmark/raw-records/decimal128.yml |   4 +-
 .../red-arrow/benchmark/raw-records/dictionary.yml |  22 +-
 ruby/red-arrow/benchmark/raw-records/int64.yml |   4 +-
 ruby/red-arrow/benchmark/raw-records/list.yml  |  10 +-
 ruby/red-arrow/benchmark/raw-records/timestamp.yml |  15 +-
 .../red-arrow/benchmark/values/boolean.yml |  38 +-
 .../{raw-records => values}/decimal128.yml |  40 +-
 .../string.yml => values/dictionary.yml}   |  47 +-
 .../red-arrow/benchmark/values/int64.yml   |  38 +-
 .../{raw-records/string.yml => values/list.yml}|  41 +-
 .../red-arrow/benchmark/values/string.yml  |  37 +-
 .../string.yml => values/timestamp.yml}|  48 +-
 ruby/red-arrow/ext/arrow/arrow.cpp |  13 +
 .../red-arrow/ext/arrow/converters.cpp |  31 +-
 ruby/red-arrow/ext/arrow/converters.hpp| 626 
 ruby/red-arrow/ext/arrow/raw-records.cpp   | 635 +
 ruby/red-arrow/ext/arrow/red-arrow.hpp |  11 +
 ruby/red-arrow/ext/arrow/values.cpp| 154 +
 ruby/red-arrow/lib/arrow/array.rb  |   4 +
 ruby/red-arrow/lib/arrow/loader.rb |   8 +
 ruby/red-arrow/test/values/test-basic-arrays.rb| 284 +
 .../test/values/test-dense-union-array.rb  | 487 
 ruby/red-arrow/test/values/test-list-array.rb  | 497 
 .../test/values/test-sparse-union-array.rb | 477 
 ruby/red-arrow/test/values/test-struct-array.rb| 452 +++
 25 files changed, 3169 insertions(+), 854 deletions(-)
 copy c_glib/Makefile.am => ruby/red-arrow/benchmark/values/boolean.yml (61%)
 copy ruby/red-arrow/benchmark/{raw-records => values}/decimal128.yml (59%)
 copy ruby/red-arrow/benchmark/{raw-records/string.yml => 
values/dictionary.yml} (59%)
 copy c_glib/Makefile.am => ruby/red-arrow/benchmark/values/int64.yml (61%)
 copy ruby/red-arrow/benchmark/{raw-records/string.yml => values/list.yml} (60%)
 copy c_glib/Makefile.am => ruby/red-arrow/benchmark/values/string.yml (62%)
 copy ruby/red-arrow/benchmark/{raw-records/string.yml => values/timestamp.yml} 
(59%)
 copy c_glib/arrow-glib/internal-index.hpp => 
ruby/red-arrow/ext/arrow/converters.cpp (50%)
 create mode 100644 ruby/red-arrow/ext/arrow/converters.hpp
 create mode 100644 ruby/red-arrow/ext/arrow/values.cpp
 create mode 100644 ruby/red-arrow/test/values/test-basic-arrays.rb
 create mode 100644 ruby/red-arrow/test/values/test-dense-union-array.rb
 create mode 100644 ruby/red-arrow/test/values/test-list-array.rb
 create mode 100644 ruby/red-arrow/test/values/test-sparse-union-array.rb
 create mode 100644 ruby/red-arrow/test/values/test-struct-array.rb

[arrow] branch master updated (e0fa3d1 -> 0f9c4a9)

2019-08-25 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from e0fa3d1  ARROW-6238: [C++][Dataset] Implement SimpleDataSource, 
SimpleDataFragment and SimpleScanTask
 add 0f9c4a9  ARROW-6350: [Ruby] Remove Arrow::Struct and use Hash instead

No new revisions were added by this update.

Summary of changes:
 ruby/red-arrow/lib/arrow/struct-array-builder.rb |  8 +--
 ruby/red-arrow/lib/arrow/struct-array.rb | 24 +++
 ruby/red-arrow/lib/arrow/struct.rb   | 79 ---
 ruby/red-arrow/test/test-struct-array-builder.rb |  8 +--
 ruby/red-arrow/test/test-struct-array.rb |  8 +--
 ruby/red-arrow/test/test-struct.rb   | 81 
 6 files changed, 22 insertions(+), 186 deletions(-)
 delete mode 100644 ruby/red-arrow/lib/arrow/struct.rb
 delete mode 100644 ruby/red-arrow/test/test-struct.rb

[arrow] branch master updated (3564c87 -> a4ef8c6)

2019-08-17 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 3564c87  ARROW-6270: [C++] check buffer_index bounds in 
IpcComponentSource.GetBuffer
 add a4ef8c6  ARROW-6267: [Ruby] Add Arrow::Time for 
Arrow::Time{32,64}DataType value

No new revisions were added by this update.

Summary of changes:
 ruby/red-arrow/ext/arrow/arrow.cpp |  21 ++
 ruby/red-arrow/ext/arrow/raw-records.cpp   |  21 +-
 ruby/red-arrow/ext/arrow/red-arrow.hpp |  29 ++-
 ruby/red-arrow/lib/arrow/array-builder.rb  |  25 +-
 ruby/red-arrow/lib/arrow/array.rb  |   5 +
 ruby/red-arrow/lib/arrow/csv-loader.rb |   4 +-
 ruby/red-arrow/lib/arrow/date64-array-builder.rb   |   4 +-
 ruby/red-arrow/lib/arrow/date64-array.rb   |   2 +-
 ruby/red-arrow/lib/arrow/loader.rb |   5 +
 ruby/red-arrow/lib/arrow/table-table-formatter.rb  |   4 +-
 ruby/red-arrow/lib/arrow/time.rb   | 159 
 ruby/red-arrow/lib/arrow/time32-array-builder.rb   |  10 +
 .../arrow/{decimal128-array.rb => time32-array.rb} |   8 +-
 ruby/red-arrow/lib/arrow/time64-array-builder.rb   |  10 +
 .../arrow/{decimal128-array.rb => time64-array.rb} |   8 +-
 .../red-arrow/lib/arrow/timestamp-array-builder.rb |   2 +-
 ruby/red-arrow/lib/arrow/timestamp-array.rb|  24 +-
 .../test/raw-records/test-basic-arrays.rb  |  24 +-
 .../test/raw-records/test-dense-union-array.rb |  15 +-
 ruby/red-arrow/test/raw-records/test-list-array.rb |  30 ++-
 .../test/raw-records/test-sparse-union-array.rb|  16 +-
 .../test/raw-records/test-struct-array.rb  |  15 +-
 ruby/red-arrow/test/test-time.rb   | 288 +
 ruby/red-arrow/test/test-time32-array.rb   |  79 --
 ruby/red-arrow/test/test-time64-array.rb   |  79 --
 25 files changed, 777 insertions(+), 110 deletions(-)
 create mode 100644 ruby/red-arrow/lib/arrow/time.rb
 copy ruby/red-arrow/lib/arrow/{decimal128-array.rb => time32-array.rb} (87%)
 copy ruby/red-arrow/lib/arrow/{decimal128-array.rb => time64-array.rb} (87%)
 create mode 100644 ruby/red-arrow/test/test-time.rb

[arrow] branch master updated (3cc12ab -> 4e51f98)

2019-08-15 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 3cc12ab  ARROW-6172 [Java] Provide benchmarks to set IntVector with 
different methods
 add 4e51f98  ARROW-6240: [Ruby] Arrow::Decimal128Array#get_value returns 
BigDecimal

No new revisions were added by this update.

Summary of changes:
 .../lib/arrow/{tensor.rb => decimal128-array.rb}   |  6 +++---
 ruby/red-arrow/lib/arrow/loader.rb |  6 +-
 .../test/test-decimal128-array-builder.rb  | 22 +++---
 ruby/red-arrow/test/test-decimal128-array.rb   |  8 
 4 files changed, 23 insertions(+), 19 deletions(-)
 copy ruby/red-arrow/lib/arrow/{tensor.rb => decimal128-array.rb} (91%)

[arrow] branch master updated: ARROW-6197: [GLib] Add garrow_decimal128_rescale()

2019-08-11 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 803c818  ARROW-6197: [GLib] Add garrow_decimal128_rescale()
803c818 is described below

commit 803c818e1f50161c2e0962ac5437d2b29125edc7
Author: Sutou Kouhei 
AuthorDate: Mon Aug 12 06:59:53 2019 +0900

ARROW-6197: [GLib] Add garrow_decimal128_rescale()

Closes #5057 from kou/glib-decimal128-rescale and squashes the following 
commits:

487b167a4   Add garrow_decimal128_rescale()

Authored-by: Sutou Kouhei 
Signed-off-by: Yosuke Shiro 
---
 c_glib/arrow-glib/decimal128.cpp | 33 -
 c_glib/arrow-glib/decimal128.h   |  6 ++
 c_glib/test/test-decimal128.rb   | 18 +-
 3 files changed, 55 insertions(+), 2 deletions(-)

diff --git a/c_glib/arrow-glib/decimal128.cpp b/c_glib/arrow-glib/decimal128.cpp
index 32bdf5f..92d5ebd 100644
--- a/c_glib/arrow-glib/decimal128.cpp
+++ b/c_glib/arrow-glib/decimal128.cpp
@@ -420,7 +420,7 @@ garrow_decimal128_divide(GArrowDecimal128 *left,
 arrow_decimal_left->Divide(*arrow_decimal_right,
_result_raw,
_remainder_raw);
-  if (garrow_error_check(error, status, "[decimal][divide]")) {
+  if (garrow_error_check(error, status, "[decimal128][divide]")) {
 if (remainder) {
   auto arrow_remainder =
 std::make_shared(arrow_remainder_raw);
@@ -436,6 +436,37 @@ garrow_decimal128_divide(GArrowDecimal128 *left,
   }
 }
 
+/**
+ * garrow_decimal128_rescale:
+ * @decimal: A #GArrowDecimal128.
+ * @original_scale: A scale to be converted from.
+ * @new_scale: A scale to be converted to.
+ * @error: (nullable): Return location for a #GError or %NULL.
+ *
+ * Returns: (nullable) (transfer full): The rescaled decimal or %NULL on error.
+ *
+ * Since: 0.15.0
+ */
+GArrowDecimal128 *
+garrow_decimal128_rescale(GArrowDecimal128 *decimal,
+  gint32 original_scale,
+  gint32 new_scale,
+  GError **error)
+{
+  auto arrow_decimal = garrow_decimal128_get_raw(decimal);
+  arrow::Decimal128 arrow_rescaled_decimal_raw;
+  auto status = arrow_decimal->Rescale(original_scale,
+   new_scale,
+   _rescaled_decimal_raw);
+  if (garrow_error_check(error, status, "[decimal128][rescale]")) {
+auto arrow_rescaled_decimal =
+  std::make_shared(arrow_rescaled_decimal_raw);
+return garrow_decimal128_new_raw(_rescaled_decimal);
+  } else {
+return NULL;
+  }
+}
+
 G_END_DECLS
 
 GArrowDecimal128 *
diff --git a/c_glib/arrow-glib/decimal128.h b/c_glib/arrow-glib/decimal128.h
index e7601a4..7079737 100644
--- a/c_glib/arrow-glib/decimal128.h
+++ b/c_glib/arrow-glib/decimal128.h
@@ -72,5 +72,11 @@ GArrowDecimal128 *garrow_decimal128_divide(GArrowDecimal128 
*left,
GArrowDecimal128 *right,
GArrowDecimal128 **remainder,
GError **error);
+GARROW_AVAILABLE_IN_0_15
+GArrowDecimal128 *
+garrow_decimal128_rescale(GArrowDecimal128 *decimal,
+  gint32 original_scale,
+  gint32 new_scale,
+  GError **error);
 
 G_END_DECLS
diff --git a/c_glib/test/test-decimal128.rb b/c_glib/test/test-decimal128.rb
index de9453c..0e4bc82 100644
--- a/c_glib/test/test-decimal128.rb
+++ b/c_glib/test/test-decimal128.rb
@@ -101,7 +101,7 @@ class TestDecimal128 < Test::Unit::TestCase
 decimal1 = Arrow::Decimal128.new(23423445)
 decimal2 = Arrow::Decimal128.new(0)
 message =
-  "[decimal][divide]: Invalid: Division by 0 in Decimal128"
+  "[decimal128][divide]: Invalid: Division by 0 in Decimal128"
 assert_raise(Arrow::Error::Invalid.new(message)) do
   decimal1.divide(decimal2)
 end
@@ -203,4 +203,20 @@ class TestDecimal128 < Test::Unit::TestCase
decimal >= decimal
  ])
   end
+
+  def test_rescale
+decimal = Arrow::Decimal128.new(10)
+assert_equal(Arrow::Decimal128.new(1000),
+ decimal.rescale(1, 3))
+  end
+
+  def test_rescale_fail
+decimal = Arrow::Decimal128.new(10)
+message =
+  "[decimal128][rescale]: Invalid: " +
+  "Rescaling decimal value would cause data loss"
+assert_raise(Arrow::Error::Invalid.new(message)) do
+  decimal.rescale(1, -1)
+end
+  end
 end

[arrow] branch master updated (34dd3ed -> 102ba09)

2019-08-11 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 34dd3ed  ARROW-6097: [Java] Avro adapter implement unions type
 add 102ba09  ARROW-6196: [Ruby] Add support for building 
Arrow::TimeNNArray by .new

No new revisions were added by this update.

Summary of changes:
 ruby/red-arrow/lib/arrow/loader.rb |  2 ++
 ...compression-type.rb => time32-array-builder.rb} | 26 ---
 ...compression-type.rb => time64-array-builder.rb} | 26 ---
 ...st-time32-data-type.rb => test-time32-array.rb} | 38 --
 ...st-time32-data-type.rb => test-time64-array.rb} | 38 --
 5 files changed, 72 insertions(+), 58 deletions(-)
 copy ruby/red-arrow/lib/arrow/{compression-type.rb => time32-array-builder.rb} 
(67%)
 copy ruby/red-arrow/lib/arrow/{compression-type.rb => time64-array-builder.rb} 
(67%)
 copy ruby/red-arrow/test/{test-time32-data-type.rb => test-time32-array.rb} 
(56%)
 copy ruby/red-arrow/test/{test-time32-data-type.rb => test-time64-array.rb} 
(56%)

[arrow] branch master updated: ARROW-6192: [GLib] Use the same SO version as C++

2019-08-10 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 53c299e  ARROW-6192: [GLib] Use the same SO version as C++
53c299e is described below

commit 53c299e01aa82d7380022fa07155b125dca398fe
Author: Sutou Kouhei 
AuthorDate: Sun Aug 11 07:54:16 2019 +0900

ARROW-6192: [GLib] Use the same SO version as C++

Closes #5051 from kou/glib-use-the-same-so-versions-as-cpp and squashes the 
following commits:

5d827fc77   Use the same SO version as C++

Authored-by: Sutou Kouhei 
Signed-off-by: Yosuke Shiro 
---
 c_glib/configure.ac | 2 +-
 c_glib/meson.build  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/c_glib/configure.ac b/c_glib/configure.ac
index ba3909c..66f88c0 100644
--- a/c_glib/configure.ac
+++ b/c_glib/configure.ac
@@ -66,7 +66,7 @@ AC_MSG_RESULT([$os_macos])
 AM_CONDITIONAL(OS_MACOS, test "$os_macos" = "yes")
 
 LT_INIT
-LT_CURRENT=${GARROW_VERSION_MINOR}
+LT_CURRENT=$(expr ${GARROW_VERSION_MAJOR} \* 100 + ${GARROW_VERSION_MINOR})
 LT_REVISION=${GARROW_VERSION_MICRO}
 LT_AGE=0
 LT_VERSION_INFO="\$(LT_CURRENT):\$(LT_REVISION):\$(LT_AGE)"
diff --git a/c_glib/meson.build b/c_glib/meson.build
index abb39da..319d698 100644
--- a/c_glib/meson.build
+++ b/c_glib/meson.build
@@ -36,7 +36,7 @@ version_minor = version_numbers[1].to_int()
 version_micro = version_numbers[2].to_int()
 
 api_version = '1.0'
-so_version = version_minor
+so_version = version_major * 100 + version_minor
 so_version_patch = version_micro
 library_version = '@0@.@1@.@2@'.format(so_version, so_version_patch, 0)

[arrow] branch master updated: ARROW-5943: [GLib][Gandiva] Add support for function aliases

2019-07-14 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new f05f647  ARROW-5943: [GLib][Gandiva] Add support for function aliases
f05f647 is described below

commit f05f6474c8736dde342ccfe716c11182fecbf773
Author: Sutou Kouhei 
AuthorDate: Mon Jul 15 10:07:45 2019 +0900

ARROW-5943: [GLib][Gandiva] Add support for function aliases

This is follow-up for ARROW-5892: https://github.com/apache/arrow/pull/4835

Author: Sutou Kouhei 

Closes #4874 from kou/glib-gandiva-function-aliases and squashes the 
following commits:

306d06450  Add symbol list for 1.0.0
5f173d646   Add support for function aliases
---
 c_glib/doc/gandiva-glib/gandiva-glib-docs.xml |  4 ++
 c_glib/gandiva-glib/native-function.cpp   | 58 +++
 c_glib/gandiva-glib/native-function.h |  3 +-
 c_glib/test/gandiva/test-function-registry.rb |  2 +-
 c_glib/test/gandiva/test-native-function.rb   | 15 ---
 c_glib/test/helper/data-type.rb   |  4 ++
 6 files changed, 43 insertions(+), 43 deletions(-)

diff --git a/c_glib/doc/gandiva-glib/gandiva-glib-docs.xml 
b/c_glib/doc/gandiva-glib/gandiva-glib-docs.xml
index 81bbb08..ac9f686 100644
--- a/c_glib/doc/gandiva-glib/gandiva-glib-docs.xml
+++ b/c_glib/doc/gandiva-glib/gandiva-glib-docs.xml
@@ -84,6 +84,10 @@
 Index of deprecated API
 
   
+  
+Index of new symbols in 1.0.0
+
+  
   
 Index of new symbols in 0.14.0
 
diff --git a/c_glib/gandiva-glib/native-function.cpp 
b/c_glib/gandiva-glib/native-function.cpp
index f2f15ac..5ff265f 100644
--- a/c_glib/gandiva-glib/native-function.cpp
+++ b/c_glib/gandiva-glib/native-function.cpp
@@ -93,46 +93,25 @@ 
ggandiva_native_function_class_init(GGandivaNativeFunctionClass *klass)
 }
 
 /**
- * ggandiva_native_function_get_signature:
+ * ggandiva_native_function_get_signatures:
  * @native_function: A #GGandivaNativeFunction.
  *
- * Returns: (transfer full): A #GGandivaFunctionSignature that represents
- *   the signature of the native function.
+ * Returns: (element-type GGandivaFunctionSignature) (transfer full):
+ *   A list of #GGandivaFunctionSignature supported by the native function.
  *
- * Since: 0.14.0
- */
-GGandivaFunctionSignature *
-ggandiva_native_function_get_signature(GGandivaNativeFunction *native_function)
-{
-  auto gandiva_native_function =
-ggandiva_native_function_get_raw(native_function);
-  auto _function_signature = 
gandiva_native_function->signatures().front();
-  return ggandiva_function_signature_new_raw(_function_signature);
-}
-
-/**
- * ggandiva_native_function_get_all_signatures:
- * @native_function: A #GGandivaNativeFunction.
- *
- * Returns: (transfer full): A List of #GGandivaFunctionSignature supported by
- * the native function.
- *
- * Since: ??
+ * Since: 1.0.0
  */
 GList *
-ggandiva_native_function_get_all_signatures(GGandivaNativeFunction 
*native_function)
+ggandiva_native_function_get_signatures(GGandivaNativeFunction 
*native_function)
 {
   auto gandiva_native_function =
-  ggandiva_native_function_get_raw(native_function);
-  
-  GList *function_signatures = nullptr;
-  for (auto& function_sig_raw : gandiva_native_function->signatures()) {
-auto function_sig = ggandiva_function_signature_new_raw(_sig_raw);
-function_signatures = g_list_prepend(function_signatures, function_sig);
+ggandiva_native_function_get_raw(native_function);
+  GList *signatures = nullptr;
+  for (auto _signature : gandiva_native_function->signatures()) {
+auto signature = ggandiva_function_signature_new_raw(_signature);
+signatures = g_list_prepend(signatures, signature);
   }
-  function_signatures = g_list_reverse(function_signatures);
-  
-  return function_signatures;
+  return g_list_reverse(signatures);
 }
 
 /**
@@ -160,7 +139,7 @@ ggandiva_native_function_equal(GGandivaNativeFunction 
*native_function,
  * @native_function: A #GGandivaNativeFunction.
  *
  * Returns: (transfer full):
- *   The string representation of the signature of the native function.
+ *   The string representation of the signatures of the native function.
  *   It should be freed with g_free() when no longer needed.
  *
  * Since: 0.14.0
@@ -170,8 +149,17 @@ ggandiva_native_function_to_string(GGandivaNativeFunction 
*native_function)
 {
   auto gandiva_native_function =
 ggandiva_native_function_get_raw(native_function);
-  auto gandiva_function_signature = 
gandiva_native_function->signatures().front();
-  return g_strdup(gandiva_function_signature.ToString().c_str());
+  auto string = g_string_new(NULL);
+  for (auto _signature : gandiva_native_function->signatures()) {
+if (string->len > 0) {
+  g_string_append(string, ", ");
+}
+const auto _string = gandiva_signature.ToString

[arrow] branch master updated: ARROW-5939: [Release] Add support for generating vote email template separately

2019-07-13 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 5c23e98  ARROW-5939: [Release] Add support for generating vote email 
template separately
5c23e98 is described below

commit 5c23e98f0e645e621cad9e86ed5cadf15366
Author: Sutou Kouhei 
AuthorDate: Sat Jul 13 18:04:50 2019 +0900

ARROW-5939: [Release] Add support for generating vote email template 
separately

Author: Sutou Kouhei 

Closes #4870 from kou/release-source-vote and squashes the following 
commits:

b98b7388d   Add support for generating vote email template 
separately
---
 dev/release/02-source-test.rb | 58 +---
 dev/release/02-source.sh  | 62 +++
 dev/release/test-helper.rb|  4 +++
 3 files changed, 91 insertions(+), 33 deletions(-)

diff --git a/dev/release/02-source-test.rb b/dev/release/02-source-test.rb
index cf3c887..ac911cd 100644
--- a/dev/release/02-source-test.rb
+++ b/dev/release/02-source-test.rb
@@ -15,8 +15,6 @@
 # specific language governing permissions and limitations
 # under the License.
 
-require "rexml/document"
-
 class SourceTest < Test::Unit::TestCase
   include GitRunnable
   include VersionDetectable
@@ -42,13 +40,14 @@ class SourceTest < Test::Unit::TestCase
 targets.each do |target|
   env["SOURCE_#{target}"] = "1"
 end
-sh(env, @script, @release_version, "0")
+output = sh(env, @script, @release_version, "0")
 sh("tar", "xf", "#{@tag_name}.tar.gz")
+output
   end
 
   def test_symbolic_links
 source
-Dir.chdir("#{@tag_name}") do
+Dir.chdir(@tag_name) do
   assert_equal([],
Find.find(".").find_all {|path| File.symlink?(path)})
 end
@@ -98,4 +97,55 @@ class SourceTest < Test::Unit::TestCase
Dir.glob("dist/pyarrow-*.tar.gz"))
 end
   end
+
+  def test_vote
+jira_url = "https://issues.apache.org/jira;
+jql_conditions = [
+  "project = ARROW",
+  "status in (Resolved, Closed)",
+  "fixVersion = #{@release_version}",
+]
+jql = jql_conditions.join(" AND ")
+n_resolved_issues = nil
+open("#{jira_url}/rest/api/2/search?jql=#{CGI.escape(jql)}") do |response|
+  n_resolved_issues = JSON.parse(response.read)["total"]
+end
+output = source("VOTE")
+assert_equal(<<-VOTE.strip, output[/^-+$(.+?)^-+$/m, 1].strip)
+To: d...@arrow.apache.org
+Subject: [VOTE] Release Apache Arrow #{@release_version} - RC0
+
+Hi,
+
+I would like to propose the following release candidate (RC0) of Apache
+Arrow version #{@release_version}. This is a release consiting of 
#{n_resolved_issues}
+resolved JIRA issues[1].
+
+This release candidate is based on commit:
+#{@current_commit} [2]
+
+The source release rc0 is hosted at [3].
+The binary artifacts are hosted at [4][5][6][7].
+The changelog is located at [8].
+
+Please download, verify checksums and signatures, run the unit tests,
+and vote on the release. See [9] for how to validate a release candidate.
+
+The vote will be open for at least 72 hours.
+
+[ ] +1 Release this as Apache Arrow #{@release_version}
+[ ] +0
+[ ] -1 Do not release this as Apache Arrow #{@release_version} because...
+
+[1]: 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20%28Resolved%2C%20Closed%29%20AND%20fixVersion%20%3D%20#{@release_version}
+[2]: https://github.com/apache/arrow/tree/#{@current_commit}
+[3]: 
https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-#{@release_version}-rc0
+[4]: https://bintray.com/apache/arrow/centos-rc/#{@release_version}-rc0
+[5]: https://bintray.com/apache/arrow/debian-rc/#{@release_version}-rc0
+[6]: https://bintray.com/apache/arrow/python-rc/#{@release_version}-rc0
+[7]: https://bintray.com/apache/arrow/ubuntu-rc/#{@release_version}-rc0
+[8]: https://github.com/apache/arrow/blob/#{@current_commit}/CHANGELOG.md
+[9]: 
https://cwiki.apache.org/confluence/display/ARROW/How+to+Verify+Release+Candidates
+VOTE
+  end
 end
diff --git a/dev/release/02-source.sh b/dev/release/02-source.sh
index ca192a8..f70503a 100755
--- a/dev/release/02-source.sh
+++ b/dev/release/02-source.sh
@@ -21,9 +21,10 @@
 set -e
 
 : ${SOURCE_DEFAULT:=1}
-: ${SOURCE_UPLOAD:=${SOURCE_DEFAULT}}
 : ${SOURCE_GLIB:=${SOURCE_DEFAULT}}
 : ${SOURCE_RAT:=${SOURCE_DEFAULT}}
+: ${SOURCE_UPLOAD:=${SOURCE_DEFAULT}}
+: ${SOURCE_VOTE:=${SOURCE_DEFAULT}}
 
 SOURCE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 SOURCE_TOP_DIR="$(cd "${SOURCE_DIR}/../../" && pwd)"
@@ -

[arrow] branch master updated (3533213 -> f01b17b)

2019-06-23 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 3533213  ARROW-5693: [Go] skip IPC integration tests for Decimal128
 add f01b17b  ARROW-5092: [C#] Create a dummy .git directory to download 
the source files from GitHub with Source Link

No new revisions were added by this update.

Summary of changes:
 ci/travis_release_test.sh   |  10 ++
 csharp/src/Apache.Arrow/Apache.Arrow.csproj |   2 +-
 dev/release/00-prepare-test.rb  |   9 +-
 dev/release/02-source-test.rb   | 103 +
 dev/release/02-source.sh| 218 
 dev/release/source/build.sh |   2 +-
 dev/release/test-helper.rb  |  13 ++
 python/setup.py |   2 +-
 8 files changed, 254 insertions(+), 105 deletions(-)
 create mode 100644 dev/release/02-source-test.rb

[arrow] branch master updated: ARROW-5447: [Ruby] Ensure flushing test gz file

2019-06-17 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 9425831  ARROW-5447: [Ruby] Ensure flushing test gz file
9425831 is described below

commit 9425831dfaf854b5f6e26af1b23afe60f883af74
Author: Sutou Kouhei 
AuthorDate: Mon Jun 17 19:43:40 2019 +0900

ARROW-5447: [Ruby] Ensure flushing test gz file

Author: Sutou Kouhei 

Closes #4584 from kou/ruby-test-robust and squashes the following commits:

c1bdcaefe  Run MinGW build when Ruby codes are changed
dc498bd49   Ensure flushing test gz file
---
 appveyor.yml  | 3 ++-
 ci/appveyor-filter-changes.bat| 6 +++---
 ruby/red-arrow/test/test-table.rb | 5 +++--
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/appveyor.yml b/appveyor.yml
index 3e0e645..cefa28e 100644
--- a/appveyor.yml
+++ b/appveyor.yml
@@ -19,7 +19,7 @@
 os: Visual Studio 2015
 
 only_commits:
-  # Skip commits not related to Python, C++, C#, Go or Rust
+  # Skip commits not related to Python, C++, C#, Go, Ruby or Rust
   files:
 - appveyor.yml
 - c_glib/
@@ -29,6 +29,7 @@ only_commits:
 - format/
 - go/
 - python/
+- ruby/
 - rust/
 
 cache:
diff --git a/ci/appveyor-filter-changes.bat b/ci/appveyor-filter-changes.bat
index 1e82ecd..04da517 100644
--- a/ci/appveyor-filter-changes.bat
+++ b/ci/appveyor-filter-changes.bat
@@ -22,10 +22,10 @@ if "%JOB%" == "Rust" (
 echo ===
 appveyor exit
 )
-) else if "%JOB%" == "MinGW" (
-if "%ARROW_CI_GLIB_AFFECTED%" == "0" (
+) else if "%JOB:~,5%" == "MinGW" (
+if "%ARROW_CI_RUBY_AFFECTED%" == "0" (
 echo ===
-echo === No C++, or GLib changes, exiting job
+echo === No C++, GLib or Ruby changes, exiting job
 echo ===
 appveyor exit
 )
diff --git a/ruby/red-arrow/test/test-table.rb 
b/ruby/red-arrow/test/test-table.rb
index 6af2b57..2b7a46c 100644
--- a/ruby/red-arrow/test/test-table.rb
+++ b/ruby/red-arrow/test/test-table.rb
@@ -492,7 +492,8 @@ class TableTest < Test::Unit::TestCase
 
   test("csv.gz") do
 file = Tempfile.new(["red-arrow", ".csv.gz"])
-Zlib::GzipWriter.wrap(file) do |gz|
+file.close
+Zlib::GzipWriter.open(file.path) do |gz|
   gz.write(<<-CSV)
 name,score
 alice,10
@@ -505,7 +506,7 @@ chris,-1
 0  alice  10
 1  bob29
 2  chris  -1
-  TABLE
+TABLE
   end
 end
   end

[arrow] branch master updated: ARROW-5553: [Ruby] Use the official packages to install Apache Arrow

2019-06-12 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new adba5bb  ARROW-5553: [Ruby] Use the official packages to install 
Apache Arrow
adba5bb is described below

commit adba5bb4d46403e2d3df431375b6c654ad0570f6
Author: Sutou Kouhei 
AuthorDate: Wed Jun 12 21:00:40 2019 +0900

ARROW-5553: [Ruby] Use the official packages to install Apache Arrow

packages.red-data-tools.org is deprecated.

Author: Sutou Kouhei 

Closes #4526 from kou/ruby-update-package-information and squashes the 
following commits:

898ef4e54   Use the official packages to install Apache Arrow
---
 ruby/red-arrow-cuda/README.md | 4 +---
 ruby/red-arrow/README.md  | 4 +---
 ruby/red-gandiva/README.md| 4 +---
 ruby/red-parquet/README.md| 4 +---
 ruby/red-plasma/README.md | 4 +---
 5 files changed, 5 insertions(+), 15 deletions(-)

diff --git a/ruby/red-arrow-cuda/README.md b/ruby/red-arrow-cuda/README.md
index 76fa51c..f05e664 100644
--- a/ruby/red-arrow-cuda/README.md
+++ b/ruby/red-arrow-cuda/README.md
@@ -33,9 +33,7 @@ gobject-introspection gem is a Ruby bindings of GObject 
Introspection. Red Arrow
 
 ## Install
 
-Install Apache Arrow CUDA GLib before install Red Arrow CUDA. Use 
[packages.red-data-tools.org](https://github.com/red-data-tools/packages.red-data-tools.org)
 for installing Apache Arrow CUDA GLib.
-
-Note that the Apache Arrow CUDA GLib packages are "unofficial". "Official" 
packages will be released in the future.
+Install Apache Arrow CUDA GLib before install Red Arrow CUDA. Install Apache 
Arrow GLib before install Red Arrow. See [Apache Arrow install 
document](https://arrow.apache.org/install/) for details.
 
 Install Red Arrow CUDA after you install Apache Arrow CUDA GLib:
 
diff --git a/ruby/red-arrow/README.md b/ruby/red-arrow/README.md
index 95ec396..20ca83f 100644
--- a/ruby/red-arrow/README.md
+++ b/ruby/red-arrow/README.md
@@ -33,9 +33,7 @@ gobject-introspection gem is a Ruby bindings of GObject 
Introspection. Red Arrow
 
 ## Install
 
-Install Apache Arrow GLib before install Red Arrow. Use 
[packages.red-data-tools.org](https://github.com/red-data-tools/packages.red-data-tools.org)
 for installing Apache Arrow GLib.
-
-Note that the Apache Arrow GLib packages are "unofficial". "Official" packages 
will be released in the future.
+Install Apache Arrow GLib before install Red Arrow. See [Apache Arrow install 
document](https://arrow.apache.org/install/) for details.
 
 Install Red Arrow after you install Apache Arrow GLib:
 
diff --git a/ruby/red-gandiva/README.md b/ruby/red-gandiva/README.md
index d6ab944..91174ee 100644
--- a/ruby/red-gandiva/README.md
+++ b/ruby/red-gandiva/README.md
@@ -33,9 +33,7 @@ gobject-introspection gem is a Ruby bindings of GObject 
Introspection. Red Gandi
 
 ## Install
 
-Install Gandiva GLib before install Red Gandiva. Use 
[packages.red-data-tools.org](https://github.com/red-data-tools/packages.red-data-tools.org)
 for installing Gandiva GLib.
-
-Note that the Gandiva GLib packages are "unofficial". "Official" packages will 
be released in the future.
+Install Gandiva GLib before install Red Gandiva. See [Apache Arrow install 
document](https://arrow.apache.org/install/) for details.
 
 Install Red Gandiva after you install Gandiva GLib:
 
diff --git a/ruby/red-parquet/README.md b/ruby/red-parquet/README.md
index 4fb8438..434dab9 100644
--- a/ruby/red-parquet/README.md
+++ b/ruby/red-parquet/README.md
@@ -33,9 +33,7 @@ gobject-introspection gem is a Ruby bindings of GObject 
Introspection. Red Parqu
 
 ## Install
 
-Install Apache Parquet GLib before install Red Parquet. Use 
[packages.red-data-tools.org](https://github.com/red-data-tools/packages.red-data-tools.org)
 for installing Apache Parquet GLib.
-
-Note that the Apache Parquet GLib packages are "unofficial". "Official" 
packages will be released in the future.
+Install Apache Parquet GLib before install Red Parquet. See [Apache Arrow 
install document](https://arrow.apache.org/install/) for details.
 
 Install Red Parquet after you install Apache Parquet GLib:
 
diff --git a/ruby/red-plasma/README.md b/ruby/red-plasma/README.md
index 19038af..e8939f6 100644
--- a/ruby/red-plasma/README.md
+++ b/ruby/red-plasma/README.md
@@ -33,9 +33,7 @@ gobject-introspection gem is a Ruby bindings of GObject 
Introspection. Red Plasm
 
 ## Install
 
-Install Plasma GLib before install Red Plasma. Use 
[packages.red-data-tools.org](https://github.com/red-data-tools/packages.red-data-tools.org)
 for installing Plasma GLib.
-
-Note that the Plasma GLib packages are "unofficial". "Official" packages will 
be released in the future.
+Install Plasma GLib before install Red Plasma. See [Apache Arro

[arrow] branch master updated: ARROW-5456: [GLib][Plasma] Fix dependency order on building document

2019-05-31 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 8e0e2c0  ARROW-5456: [GLib][Plasma] Fix dependency order on building 
document
8e0e2c0 is described below

commit 8e0e2c06cbb797966f66fd2bcf9043c01006be0d
Author: Sutou Kouhei 
AuthorDate: Sat Jun 1 11:38:00 2019 +0900

ARROW-5456: [GLib][Plasma] Fix dependency order on building document

We use installed plasma-glib with allow_glib -> plasma_glib order.
We must use building plasma-glib instead of installed plasma-glib.

Author: Sutou Kouhei 

Closes #4425 from kou/glib-plasma-fix-dependency-order and squashes the 
following commits:

b8053c19   Fix dependency order on building document
---
 c_glib/doc/plasma-glib/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/c_glib/doc/plasma-glib/meson.build 
b/c_glib/doc/plasma-glib/meson.build
index 9efc53b..490b650 100644
--- a/c_glib/doc/plasma-glib/meson.build
+++ b/c_glib/doc/plasma-glib/meson.build
@@ -53,8 +53,8 @@ source_directories = [
   join_paths(meson.build_root(), 'plasma-glib'),
 ]
 dependencies = [
-  arrow_glib,
   plasma_glib,
+  arrow_glib,
 ]
 if arrow_cuda.found()
   dependencies += [arrow_cuda_glib]

[arrow] branch master updated: ARROW-5051: [GLib][Gandiva] Don't return temporary memory

2019-03-29 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 114985c  ARROW-5051: [GLib][Gandiva] Don't return temporary memory
114985c is described below

commit 114985c3ca8570a683dbc61062ed781daa3ddc8e
Author: Kouhei Sutou 
AuthorDate: Sat Mar 30 04:47:44 2019 +0900

ARROW-5051: [GLib][Gandiva] Don't return temporary memory

Author: Kouhei Sutou 

Closes #4071 from kou/glib-gandiva-fix-string-literal-node-value and 
squashes the following commits:

4a1811b8   Don't return temporary memory
---
 c_glib/gandiva-glib/node.cpp | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/c_glib/gandiva-glib/node.cpp b/c_glib/gandiva-glib/node.cpp
index 347473b..0c47a7c 100644
--- a/c_glib/gandiva-glib/node.cpp
+++ b/c_glib/gandiva-glib/node.cpp
@@ -28,7 +28,7 @@
 #include 
 
 template 
-Type
+const Type &
 ggandiva_literal_node_get(GGandivaLiteralNode *node)
 {
   auto gandiva_literal_node =
@@ -1178,7 +1178,7 @@ ggandiva_string_literal_node_new(const gchar *value)
 const gchar *
 ggandiva_string_literal_node_get_value(GGandivaStringLiteralNode *node)
 {
-  auto value = 
ggandiva_literal_node_get(GGANDIVA_LITERAL_NODE(node));
+  auto  = 
ggandiva_literal_node_get(GGANDIVA_LITERAL_NODE(node));
   return value.c_str();
 }

[arrow] branch master updated: ARROW-4977: [Ruby] Add support for building on Windows

2019-03-23 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 87e9ec3  ARROW-4977: [Ruby] Add support for building on Windows
87e9ec3 is described below

commit 87e9ec305454609ac8664409885b6a9b93e891e2
Author: Kouhei Sutou 
AuthorDate: Sat Mar 23 21:03:21 2019 +0900

ARROW-4977: [Ruby] Add support for building on Windows

Author: Kouhei Sutou 

Closes #3994 from kou/ruby-windows and squashes the following commits:

60add6e6   Run tests on AppVeyor
d17cc34e   Add support for building on Windows
---
 ci/appveyor-cpp-build-mingw.bat|  6 ++
 ruby/red-arrow/ext/arrow/extconf.rb|  6 ++
 ruby/red-arrow/ext/arrow/red-arrow.hpp |  8 
 ruby/red-arrow/lib/arrow.rb|  1 +
 ruby/red-arrow/red-arrow.gemspec   |  2 +-
 ruby/red-arrow/test/run-test.rb| 12 +++-
 ruby/red-arrow/test/test-group.rb  |  4 ++--
 7 files changed, 31 insertions(+), 8 deletions(-)

diff --git a/ci/appveyor-cpp-build-mingw.bat b/ci/appveyor-cpp-build-mingw.bat
index cca9877..c92b8eb 100644
--- a/ci/appveyor-cpp-build-mingw.bat
+++ b/ci/appveyor-cpp-build-mingw.bat
@@ -26,6 +26,7 @@ set PKG_CONFIG_PATH=%INSTALL_DIR%\lib\pkgconfig
 set GI_TYPELIB_PATH=%INSTALL_DIR%\lib\girepository-1.0
 set ARROW_DLL_PATH=%MINGW_PREFIX%\bin
 set ARROW_DLL_PATH=%INSTALL_DIR%\bin;%ARROW_DLL_PATH%
+set ARROW_PKG_CONFIG_PATH=%PKG_CONFIG_PATH%
 
 for /f "usebackq" %%v in (`python3 -c "import sys; print('.'.join(map(str, 
sys.version_info[0:2])))"`) do (
   set PYTHON_VERSION=%%v
@@ -74,3 +75,8 @@ sed -i'' -s 's/\r//g' %C_GLIB_BUILD_DIR%/arrow-glib/version.h 
|| exit /B
 ninja -C %C_GLIB_BUILD_DIR% || exit /B
 ninja -C %C_GLIB_BUILD_DIR% install || exit /B
 ruby c_glib\test\run-test.rb || exit /B
+
+pushd ruby\red-arrow
+ruby -S bundle install
+ruby -rdevkit test\run-test.rb
+popd
diff --git a/ruby/red-arrow/ext/arrow/extconf.rb 
b/ruby/red-arrow/ext/arrow/extconf.rb
index a8b9a0b..3b53a29 100644
--- a/ruby/red-arrow/ext/arrow/extconf.rb
+++ b/ruby/red-arrow/ext/arrow/extconf.rb
@@ -18,6 +18,12 @@
 require "extpp"
 require "mkmf-gnome2"
 
+arrow_pkg_config_path = ENV["ARROW_PKG_CONFIG_PATH"]
+if arrow_pkg_config_path
+  pkg_config_paths = [arrow_pkg_config_path, ENV["PKG_CONFIG_PATH"]].compact
+  ENV["PKG_CONFIG_PATH"] = pkg_config_paths.join(File::PATH_SEPARATOR)
+end
+
 unless required_pkg_config_package("arrow",
debian: "libarrow-dev",
redhat: "arrow-devel",
diff --git a/ruby/red-arrow/ext/arrow/red-arrow.hpp 
b/ruby/red-arrow/ext/arrow/red-arrow.hpp
index 5c9b846..6386d3d 100644
--- a/ruby/red-arrow/ext/arrow/red-arrow.hpp
+++ b/ruby/red-arrow/ext/arrow/red-arrow.hpp
@@ -21,6 +21,14 @@
 
 #include 
 
+#ifdef _WIN32
+#  define gmtime_r gmtime_r_ruby_win32
+#  define localtime_r localtime_r_ruby_win32
+#  include 
+#  undef gmtime_r
+#  undef localtime_r
+#endif
+
 #include 
 #include 
 
diff --git a/ruby/red-arrow/lib/arrow.rb b/ruby/red-arrow/lib/arrow.rb
index 257bd28..8fbc537 100644
--- a/ruby/red-arrow/lib/arrow.rb
+++ b/ruby/red-arrow/lib/arrow.rb
@@ -15,6 +15,7 @@
 # specific language governing permissions and limitations
 # under the License.
 
+require "extpp/setup"
 require "gio2"
 
 require "arrow/version"
diff --git a/ruby/red-arrow/red-arrow.gemspec b/ruby/red-arrow/red-arrow.gemspec
index 317bcf2..f4fcda9 100644
--- a/ruby/red-arrow/red-arrow.gemspec
+++ b/ruby/red-arrow/red-arrow.gemspec
@@ -46,7 +46,7 @@ Gem::Specification.new do |spec|
   spec.test_files += Dir.glob("test/**/*")
   spec.extensions = ["ext/arrow/extconf.rb"]
 
-  spec.add_runtime_dependency("extpp")
+  spec.add_runtime_dependency("extpp", ">= 0.0.7")
   spec.add_runtime_dependency("gio2", ">= 3.3.6")
   spec.add_runtime_dependency("native-package-installer")
   spec.add_runtime_dependency("pkg-config")
diff --git a/ruby/red-arrow/test/run-test.rb b/ruby/red-arrow/test/run-test.rb
index 4712d49..4ed9a63 100755
--- a/ruby/red-arrow/test/run-test.rb
+++ b/ruby/red-arrow/test/run-test.rb
@@ -17,12 +17,14 @@
 # specific language governing permissions and limitations
 # under the License.
 
-ENV["TZ"] = "Asia/Tokyo"
-
 $VERBOSE = true
 
 require "pathname"
 
+(ENV["ARROW_DLL_PATH"] || "").split(File::PATH_SEPARATOR).each do |path|
+  RubyInstaller::Runtime.add_dll_directory(path)
+end
+
 base_dir = Pathname.new(__dir__).parent.expand_path
 
 lib_dir = base_dir + "lib"
@@ -33,9 +35,9 @@ make = nil
 if ENV["NO_MAKE"] != "yes"
   if ENV["MAKE&

[arrow] branch master updated: ARROW-4981: [Ruby] Add support for CSV data encoding conversion

2019-03-22 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 5b87c55  ARROW-4981: [Ruby] Add support for CSV data encoding 
conversion
5b87c55 is described below

commit 5b87c553d6c1d13280af48b919f0269f3ad686a4
Author: Kouhei Sutou 
AuthorDate: Sat Mar 23 09:39:21 2019 +0900

ARROW-4981: [Ruby] Add support for CSV data encoding conversion

Author: Kouhei Sutou 

Closes #3998 from kou/ruby-csv-encoding and squashes the following commits:

6f372ea2   Add support for CSV data encoding conversion
---
 ruby/red-arrow/lib/arrow.rb|  2 +-
 ruby/red-arrow/lib/arrow/csv-loader.rb | 34 --
 ruby/red-arrow/red-arrow.gemspec   |  2 +-
 ruby/red-arrow/test/test-csv-loader.rb | 30 ++
 4 files changed, 60 insertions(+), 8 deletions(-)

diff --git a/ruby/red-arrow/lib/arrow.rb b/ruby/red-arrow/lib/arrow.rb
index 95dabee..257bd28 100644
--- a/ruby/red-arrow/lib/arrow.rb
+++ b/ruby/red-arrow/lib/arrow.rb
@@ -15,7 +15,7 @@
 # specific language governing permissions and limitations
 # under the License.
 
-require "gobject-introspection"
+require "gio2"
 
 require "arrow/version"
 
diff --git a/ruby/red-arrow/lib/arrow/csv-loader.rb 
b/ruby/red-arrow/lib/arrow/csv-loader.rb
index bb1f419..1839590 100644
--- a/ruby/red-arrow/lib/arrow/csv-loader.rb
+++ b/ruby/red-arrow/lib/arrow/csv-loader.rb
@@ -104,6 +104,8 @@ module Arrow
   end
 when :schema
   options.add_schema(value)
+when :encoding
+  # process encoding on opening input
 else
   setter = "#{key}="
   if options.respond_to?(setter)
@@ -116,7 +118,7 @@ module Arrow
   options
 end
 
-def open_input(raw_input)
+def open_decompress_input(raw_input)
   if @compression
 codec = Codec.new(@compression)
 CompressedInputStream.open(codec, raw_input) do |input|
@@ -127,16 +129,36 @@ module Arrow
   end
 end
 
+def open_encoding_convert_stream(raw_input, )
+  encoding = @options[:encoding]
+  if encoding
+converter = Gio::CharsetConverter.new("UTF-8", encoding)
+convert_input_stream =
+  Gio::ConverterInputStream.new(raw_input, converter)
+GIOInputStream.open(convert_input_stream, )
+  else
+yield(raw_input)
+  end
+end
+
+def wrap_input(raw_input)
+  open_decompress_input(raw_input) do |input_|
+open_encoding_convert_stream(input_) do |input__|
+  yield(input__)
+end
+  end
+end
+
 def load_from_path(path)
   options = reader_options
   if options
 begin
-  MemoryMappedInputStream.open(path.to_s) do |raw_input|
-open_input(raw_input) do |input|
+  MemoryMappedInputStream.open(path) do |raw_input|
+wrap_input(raw_input) do |input|
   return CSVReader.new(input, options).read
 end
   end
-rescue Arrow::Error::Invalid
+rescue Arrow::Error::Invalid, Gio::Error
 end
   end
 
@@ -151,11 +173,11 @@ module Arrow
   if options
 begin
   BufferInputStream.open(Buffer.new(data)) do |raw_input|
-open_input(raw_input) do |input|
+wrap_input(raw_input) do |input|
   return CSVReader.new(input, options).read
 end
   end
-rescue Arrow::Error::Invalid
+rescue Arrow::Error::Invalid, Gio::Error
 end
   end
 
diff --git a/ruby/red-arrow/red-arrow.gemspec b/ruby/red-arrow/red-arrow.gemspec
index 7c6320e..317bcf2 100644
--- a/ruby/red-arrow/red-arrow.gemspec
+++ b/ruby/red-arrow/red-arrow.gemspec
@@ -47,7 +47,7 @@ Gem::Specification.new do |spec|
   spec.extensions = ["ext/arrow/extconf.rb"]
 
   spec.add_runtime_dependency("extpp")
-  spec.add_runtime_dependency("gobject-introspection", ">= 3.3.5")
+  spec.add_runtime_dependency("gio2", ">= 3.3.6")
   spec.add_runtime_dependency("native-package-installer")
   spec.add_runtime_dependency("pkg-config")
 
diff --git a/ruby/red-arrow/test/test-csv-loader.rb 
b/ruby/red-arrow/test/test-csv-loader.rb
index b5b8278..96de9c8 100644
--- a/ruby/red-arrow/test/test-csv-loader.rb
+++ b/ruby/red-arrow/test/test-csv-loader.rb
@@ -141,5 +141,35 @@ count
 4
CSV
 end
+
+test(":encoding") do
+  messages = [
+"\u3042", # U+3042 HIRAGANA LETTER A
+"\u3044", # U+3044 HIRAGANA LETTER I
+"\u3046", # U+3046 HIRAGANA LETTER U
+  ]
+  table = Arrow::Table.new(:message => Arrow::StringArray.new(messages))
+

[arrow] branch master updated: ARROW-4980: [GLib] Use GInputStream as the parent of GArrowInputStream

2019-03-22 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 72ce489  ARROW-4980: [GLib] Use GInputStream as the parent of 
GArrowInputStream
72ce489 is described below

commit 72ce4892b0f1ccbc5b7adfa530ca36af5264a339
Author: Kouhei Sutou 
AuthorDate: Sat Mar 23 07:50:25 2019 +0900

ARROW-4980: [GLib] Use GInputStream as the parent of GArrowInputStream

If we use GInputStream as the parent of GArrowInputStream, we can use
GInputStream related feature such as encoding conversion to Arrow's
input stream.

Author: Kouhei Sutou 

Closes #3997 from kou/glib-gio-input-stream and squashes the following 
commits:

57381937   Use GInputStream as the parent of GArrowInputStream
---
 c_glib/arrow-glib/input-stream.cpp  | 66 ++---
 c_glib/arrow-glib/input-stream.h|  4 +-
 c_glib/test/test-buffer-input-stream.rb | 15 
 3 files changed, 77 insertions(+), 8 deletions(-)

diff --git a/c_glib/arrow-glib/input-stream.cpp 
b/c_glib/arrow-glib/input-stream.cpp
index ab24186..459745d 100644
--- a/c_glib/arrow-glib/input-stream.cpp
+++ b/c_glib/arrow-glib/input-stream.cpp
@@ -65,8 +65,7 @@ typedef struct GArrowInputStreamPrivate_ {
 } GArrowInputStreamPrivate;
 
 enum {
-  PROP_0,
-  PROP_INPUT_STREAM
+  PROP_INPUT_STREAM = 1
 };
 
 static std::shared_ptr
@@ -100,7 +99,7 @@ 
garrow_input_stream_readable_interface_init(GArrowReadableInterface *iface)
 
 G_DEFINE_TYPE_WITH_CODE(GArrowInputStream,
 garrow_input_stream,
-G_TYPE_OBJECT,
+G_TYPE_INPUT_STREAM,
 G_ADD_PRIVATE(GArrowInputStream)
 G_IMPLEMENT_INTERFACE(GARROW_TYPE_FILE,
   
garrow_input_stream_file_interface_init)
@@ -154,6 +153,58 @@ garrow_input_stream_get_property(GObject *object,
   }
 }
 
+static gssize
+garrow_input_stream_read(GInputStream *stream,
+ void *buffer,
+ gsize count,
+ GCancellable *cancellable,
+ GError **error)
+{
+  if (g_cancellable_set_error_if_cancelled(cancellable, error)) {
+return -1;
+  }
+  auto arrow_input_stream =
+garrow_input_stream_get_raw(GARROW_INPUT_STREAM(stream));
+  int64_t n_read_bytes;
+  auto status = arrow_input_stream->Read(count, _read_bytes, buffer);
+  if (!garrow_error_check(error, status, "[input-stream][read]")) {
+return -1;
+  }
+  return n_read_bytes;
+}
+
+static gssize
+garrow_input_stream_skip(GInputStream *stream,
+ gsize count,
+ GCancellable *cancellable,
+ GError **error)
+{
+  if (g_cancellable_set_error_if_cancelled(cancellable, error)) {
+return -1;
+  }
+  auto arrow_input_stream =
+garrow_input_stream_get_raw(GARROW_INPUT_STREAM(stream));
+  auto status = arrow_input_stream->Advance(count);
+  if (!garrow_error_check(error, status, "[input-stream][skip]")) {
+return -1;
+  }
+  return count;
+}
+
+static gboolean
+garrow_input_stream_close(GInputStream *stream,
+  GCancellable *cancellable,
+  GError **error)
+{
+  if (g_cancellable_set_error_if_cancelled(cancellable, error)) {
+return FALSE;
+  }
+  auto arrow_input_stream =
+garrow_input_stream_get_raw(GARROW_INPUT_STREAM(stream));
+  auto status = arrow_input_stream->Close();
+  return garrow_error_check(error, status, "[input-stream][close]");
+}
+
 static void
 garrow_input_stream_init(GArrowInputStream *object)
 {
@@ -163,11 +214,15 @@ static void
 garrow_input_stream_class_init(GArrowInputStreamClass *klass)
 {
   auto gobject_class = G_OBJECT_CLASS(klass);
-
   gobject_class->finalize = garrow_input_stream_finalize;
   gobject_class->set_property = garrow_input_stream_set_property;
   gobject_class->get_property = garrow_input_stream_get_property;
 
+  auto input_stream_class = G_INPUT_STREAM_CLASS(klass);
+  input_stream_class->read_fn = garrow_input_stream_read;
+  input_stream_class->skip = garrow_input_stream_skip;
+  input_stream_class->close_fn = garrow_input_stream_close;
+
   GParamSpec *spec;
   spec = g_param_spec_pointer("input-stream",
   "Input stream",
@@ -356,8 +411,7 @@ typedef struct GArrowBufferInputStreamPrivate_ {
 } GArrowBufferInputStreamPrivate;
 
 enum {
-  PROP_0_,
-  PROP_BUFFER
+  PROP_BUFFER = 1,
 };
 
 G_DEFINE_TYPE_WITH_PRIVATE(GArrowBufferInputStream,
diff --git a/c_glib/arrow-glib/input-stream.h b/c_glib/arrow-glib/input-stream.h
index 745b912..8f86741 100644
--- a/c_glib/arrow-glib/input-stream.h
+++ b/c_glib/arrow-glib/

[arrow] branch master updated: ARROW-4979: [GLib] Add missing lock to garrow::GIOInputStream

2019-03-21 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 13fd813  ARROW-4979: [GLib] Add missing lock to garrow::GIOInputStream
13fd813 is described below

commit 13fd813445b4738cbebbd137490fe3c02071c04b
Author: Kouhei Sutou 
AuthorDate: Fri Mar 22 08:35:10 2019 +0900

ARROW-4979: [GLib] Add missing lock to garrow::GIOInputStream

Author: Kouhei Sutou 

Closes #3996 from kou/glib-gio-input-stream-lock and squashes the following 
commits:

0bce99c1   Add missing lock to garrow::GIOInputStream
---
 c_glib/arrow-glib/input-stream.cpp | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/c_glib/arrow-glib/input-stream.cpp 
b/c_glib/arrow-glib/input-stream.cpp
index cb1fb3b..ab24186 100644
--- a/c_glib/arrow-glib/input-stream.cpp
+++ b/c_glib/arrow-glib/input-stream.cpp
@@ -33,6 +33,8 @@
 #include 
 #include 
 
+#include 
+
 G_BEGIN_DECLS
 
 /**
@@ -525,7 +527,8 @@ namespace garrow {
   class GIOInputStream : public arrow::io::RandomAccessFile {
   public:
 GIOInputStream(GInputStream *input_stream) :
-  input_stream_(input_stream) {
+  input_stream_(input_stream),
+  lock_() {
   g_object_ref(input_stream_);
 }
 
@@ -542,6 +545,7 @@ namespace garrow {
 }
 
 arrow::Status Close() override {
+  std::lock_guard guard(lock_);
   GError *error = NULL;
   if (g_input_stream_close(input_stream_, NULL, )) {
 return arrow::Status::OK();
@@ -568,6 +572,7 @@ namespace garrow {
 arrow::Status Read(int64_t n_bytes,
int64_t *n_read_bytes,
void *out) override {
+  std::lock_guard guard(lock_);
   GError *error = NULL;
   *n_read_bytes = g_input_stream_read(input_stream_,
   out,
@@ -600,6 +605,7 @@ namespace garrow {
   std::shared_ptr buffer;
   RETURN_NOT_OK(AllocateResizableBuffer(pool, n_bytes, ));
 
+  std::lock_guard guard(lock_);
   GError *error = NULL;
   auto n_read_bytes = g_input_stream_read(input_stream_,
   buffer->mutable_data(),
@@ -628,6 +634,7 @@ namespace garrow {
 return arrow::Status::NotImplemented(message);
   }
 
+  std::lock_guard guard(lock_);
   GError *error = NULL;
   if (g_seekable_seek(G_SEEKABLE(input_stream_),
   position,
@@ -651,6 +658,7 @@ namespace garrow {
 return arrow::Status::NotImplemented(message);
   }
 
+  std::lock_guard guard(lock_);
   auto current_position = g_seekable_tell(G_SEEKABLE(input_stream_));
   GError *error = NULL;
   if (!g_seekable_seek(G_SEEKABLE(input_stream_),
@@ -681,6 +689,7 @@ namespace garrow {
 
   private:
 GInputStream *input_stream_;
+std::mutex lock_;
   };
 };

[arrow] branch master updated: ARROW-4964: [Ruby] Add closed check if available on auto close

2019-03-19 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 6e14edc  ARROW-4964: [Ruby] Add closed check if available on auto close
6e14edc is described below

commit 6e14edcca63947d692e2ba0a94fe0704455dd53d
Author: Kouhei Sutou 
AuthorDate: Wed Mar 20 08:28:03 2019 +0900

ARROW-4964: [Ruby] Add closed check if available on auto close

Author: Kouhei Sutou 

Closes #3977 from kou/ruby-check-closed and squashes the following commits:

c4c1359d   Add closed check if available on auto close
---
 ruby/red-arrow/lib/arrow/block-closable.rb | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/ruby/red-arrow/lib/arrow/block-closable.rb 
b/ruby/red-arrow/lib/arrow/block-closable.rb
index 473ee14..ec236bd 100644
--- a/ruby/red-arrow/lib/arrow/block-closable.rb
+++ b/ruby/red-arrow/lib/arrow/block-closable.rb
@@ -24,7 +24,11 @@ module Arrow
   begin
 yield(io)
   ensure
-io.close
+if io.respond_to?(:closed?)
+  io.close unless io.closed?
+else
+  io.close
+end
   end
 end
   end

[arrow] branch master updated: ARROW-4929: [GLib] Add garrow_array_count_values()

2019-03-17 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 201a3bc  ARROW-4929: [GLib] Add garrow_array_count_values()
201a3bc is described below

commit 201a3bc9186aecd3b22529d97af30ac1fab25a3a
Author: Kouhei Sutou 
AuthorDate: Mon Mar 18 08:39:42 2019 +0900

ARROW-4929: [GLib] Add garrow_array_count_values()

Author: Kouhei Sutou 

Closes #3941 from kou/glib-count-values and squashes the following commits:

f9e3bc51  Don't use special characters in HTML
321fe28a  Move compute related code to compute.{cpp,h}
c8bd73bc  Add missing (nullable) attribute
7ff43645  Fix a typo
18b8d89e  Fix markup
95c08075   Add garrow_array_count_values()
---
 c_glib/arrow-glib/basic-array.cpp | 570 
 c_glib/arrow-glib/basic-array.h   |  66 
 c_glib/arrow-glib/composite-array.h   |  43 +--
 c_glib/arrow-glib/compute.cpp | 612 +-
 c_glib/arrow-glib/compute.h   |  71 +++-
 c_glib/doc/arrow-glib/arrow-glib-docs.xml |   5 +-
 c_glib/gandiva-glib/node.cpp  |   2 +-
 c_glib/test/test-count-values.rb  |  51 +++
 8 files changed, 738 insertions(+), 682 deletions(-)

diff --git a/c_glib/arrow-glib/basic-array.cpp 
b/c_glib/arrow-glib/basic-array.cpp
index 8f27e26..b051c97 100644
--- a/c_glib/arrow-glib/basic-array.cpp
+++ b/c_glib/arrow-glib/basic-array.cpp
@@ -24,7 +24,6 @@
 #include 
 #include 
 #include 
-#include 
 #include 
 #include 
 #include 
@@ -83,34 +82,6 @@ garrow_primitive_array_new(GArrowDataType *data_type,
   return garrow_array_new_raw(_array);
 };
 
-template 
-typename ArrowType::c_type
-garrow_numeric_array_sum(GArrowArrayType array,
- GError **error,
- const gchar *tag,
- typename ArrowType::c_type default_value)
-{
-  auto arrow_array = garrow_array_get_raw(GARROW_ARRAY(array));
-  auto memory_pool = arrow::default_memory_pool();
-  arrow::compute::FunctionContext context(memory_pool);
-  arrow::compute::Datum sum_datum;
-  auto status = arrow::compute::Sum(,
-arrow_array,
-_datum);
-  if (garrow_error_check(error, status, tag)) {
-using ScalarType = typename arrow::TypeTraits::ScalarType;
-auto arrow_numeric_scalar =
-  std::dynamic_pointer_cast(sum_datum.scalar());
-if (arrow_numeric_scalar->is_valid) {
-  return arrow_numeric_scalar->value;
-} else {
-  return default_value;
-}
-  } else {
-return default_value;
-  }
-}
-
 G_BEGIN_DECLS
 
 /**
@@ -545,177 +516,6 @@ garrow_array_to_string(GArrowArray *array, GError **error)
   }
 }
 
-/**
- * garrow_array_cast:
- * @array: A #GArrowArray.
- * @target_data_type: A #GArrowDataType of cast target data.
- * @options: (nullable): A #GArrowCastOptions.
- * @error: (nullable): Return location for a #GError or %NULL.
- *
- * Returns: (nullable) (transfer full):
- *   A newly created casted array on success, %NULL on error.
- *
- * Since: 0.7.0
- */
-GArrowArray *
-garrow_array_cast(GArrowArray *array,
-  GArrowDataType *target_data_type,
-  GArrowCastOptions *options,
-  GError **error)
-{
-  auto arrow_array = garrow_array_get_raw(array);
-  auto arrow_array_raw = arrow_array.get();
-  auto memory_pool = arrow::default_memory_pool();
-  arrow::compute::FunctionContext context(memory_pool);
-  auto arrow_target_data_type = garrow_data_type_get_raw(target_data_type);
-  std::shared_ptr arrow_casted_array;
-  arrow::Status status;
-  if (options) {
-auto arrow_options = garrow_cast_options_get_raw(options);
-status = arrow::compute::Cast(,
-  *arrow_array_raw,
-  arrow_target_data_type,
-  *arrow_options,
-  _casted_array);
-  } else {
-arrow::compute::CastOptions arrow_options;
-status = arrow::compute::Cast(,
-  *arrow_array_raw,
-  arrow_target_data_type,
-  arrow_options,
-  _casted_array);
-  }
-
-  if (!status.ok()) {
-std::stringstream message;
-message << "[array][cast] <";
-message << arrow_array->type()->ToString();
-message << "> -> <";
-message << arrow_target_data_type->ToString();
-message << ">";
-garrow_error_check(error, status, message.str().c_str());
-return NULL;
-  }
-
-  return garrow_array_new_raw(_casted_array);
-}
-
-/**
- * garrow_array_unique:
- * @array: A #GArrowArray.

[arrow] branch master updated: ARROW-4887: [GLib] Add garrow_array_count()

2019-03-16 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new c707822  ARROW-4887: [GLib] Add garrow_array_count()
c707822 is described below

commit c707822d01ccba02c823d14426188943ce7a4cdb
Author: Kouhei Sutou 
AuthorDate: Sat Mar 16 19:51:38 2019 +0900

ARROW-4887: [GLib] Add garrow_array_count()

Author: Kouhei Sutou 

Closes #3912 from kou/glib-count and squashes the following commits:

8132b636  Fix document
93b2004c   Add garrow_array_count()
---
 c_glib/arrow-glib/basic-array.cpp |  45 ++
 c_glib/arrow-glib/basic-array.h   |   5 ++
 c_glib/arrow-glib/compute.cpp | 124 +-
 c_glib/arrow-glib/compute.h   |  29 +
 c_glib/arrow-glib/compute.hpp |   5 ++
 c_glib/test/test-count.rb |  40 
 cpp/src/arrow/compute/api.h   |   1 +
 7 files changed, 246 insertions(+), 3 deletions(-)

diff --git a/c_glib/arrow-glib/basic-array.cpp 
b/c_glib/arrow-glib/basic-array.cpp
index c201f9d..79ba1ab 100644
--- a/c_glib/arrow-glib/basic-array.cpp
+++ b/c_glib/arrow-glib/basic-array.cpp
@@ -643,6 +643,51 @@ garrow_array_dictionary_encode(GArrowArray *array,
   return garrow_array_new_raw(_dictionary_encoded_array);
 }
 
+/**
+ * garrow_array_count:
+ * @array: A #GArrowArray.
+ * @options: (nullable): A #GArrowCountOptions.
+ * @error: (nullable): Return location for a #GError or %NULL.
+ *
+ * Returns: The number of target values on success. If an error is occurred,
+ *   the returned value is untrustful value.
+ *
+ * Since: 0.13.0
+ */
+gint64
+garrow_array_count(GArrowArray *array,
+   GArrowCountOptions *options,
+   GError **error)
+{
+  auto arrow_array = garrow_array_get_raw(array);
+  auto arrow_array_raw = arrow_array.get();
+  auto memory_pool = arrow::default_memory_pool();
+  arrow::compute::FunctionContext context(memory_pool);
+  arrow::compute::Datum counted_datum;
+  arrow::Status status;
+  if (options) {
+auto arrow_options = garrow_count_options_get_raw(options);
+status = arrow::compute::Count(,
+   *arrow_options,
+   *arrow_array_raw,
+   _datum);
+  } else {
+arrow::compute::CountOptions 
arrow_options(arrow::compute::CountOptions::COUNT_ALL);
+status = arrow::compute::Count(,
+   arrow_options,
+   *arrow_array_raw,
+   _datum);
+  }
+
+  if (garrow_error_check(error, status, "[array][count]")) {
+using ScalarType = typename 
arrow::TypeTraits::ScalarType;
+auto counted_scalar = 
std::dynamic_pointer_cast(counted_datum.scalar());
+return counted_scalar->value;
+  } else {
+return 0;
+  }
+}
+
 
 G_DEFINE_TYPE(GArrowNullArray,
   garrow_null_array,
diff --git a/c_glib/arrow-glib/basic-array.h b/c_glib/arrow-glib/basic-array.h
index 1dde2f2..c91de5a 100644
--- a/c_glib/arrow-glib/basic-array.h
+++ b/c_glib/arrow-glib/basic-array.h
@@ -70,6 +70,11 @@ GArrowArray   *garrow_array_unique  (GArrowArray *array,
  GError **error);
 GArrowArray   *garrow_array_dictionary_encode(GArrowArray *array,
   GError **error);
+GARROW_AVAILABLE_IN_0_13
+gint64
+garrow_array_count(GArrowArray *array,
+   GArrowCountOptions *options,
+   GError **error);
 
 #define GARROW_TYPE_NULL_ARRAY  \
   (garrow_null_array_get_type())
diff --git a/c_glib/arrow-glib/compute.cpp b/c_glib/arrow-glib/compute.cpp
index a9f6721..3dd226e 100644
--- a/c_glib/arrow-glib/compute.cpp
+++ b/c_glib/arrow-glib/compute.cpp
@@ -22,6 +22,7 @@
 #endif
 
 #include 
+#include 
 
 G_BEGIN_DECLS
 
@@ -31,7 +32,9 @@ G_BEGIN_DECLS
  * @title: Classes for computation
  * @include: arrow-glib/arrow-glib.h
  *
- * #GArrowCastOptions is a class to custom garrow_array_cast().
+ * #GArrowCastOptions is a class to customize garrow_array_cast().
+ *
+ * #GArrowCountOptions is a class to customize garrow_array_count().
  */
 
 typedef struct GArrowCastOptionsPrivate_ {
@@ -39,8 +42,7 @@ typedef struct GArrowCastOptionsPrivate_ {
 } GArrowCastOptionsPrivate;
 
 enum {
-  PROP_0,
-  PROP_ALLOW_INT_OVERFLOW,
+  PROP_ALLOW_INT_OVERFLOW = 1,
   PROP_ALLOW_TIME_TRUNCATE,
   PROP_ALLOW_FLOAT_TRUNCATE,
   PROP_ALLOW_INVALID_UTF8,
@@ -194,6 +196,105 @@ garrow_cast_options_new(void)
   return GARROW_CAST_OPTIONS(cast_options);
 }
 
+
+typedef struct GArrowCountOptionsPrivate_ {
+  arrow::compute::CountOptions options;
+} GArrowCountOptionsPrivate;
+
+enum {
+  PROP_MODE = 1,
+};
+
+G_DEFINE_TYPE_WITH_PRIVATE(GArro

[arrow] branch master updated: ARROW-4862: [GLib] Add GArrowCastOptions::allow-invalid-utf8 property

2019-03-14 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 99a47ab  ARROW-4862: [GLib] Add GArrowCastOptions::allow-invalid-utf8 
property
99a47ab is described below

commit 99a47ab1f8d3a89e5f49006f072c9fba276858e1
Author: Kouhei Sutou 
AuthorDate: Fri Mar 15 09:03:11 2019 +0900

ARROW-4862: [GLib] Add GArrowCastOptions::allow-invalid-utf8 property

Author: Kouhei Sutou 

Closes #3894 from kou/glib-cast-options-allow-invalid-utf8 and squashes the 
following commits:

9fc06744   Add GArrowCastOptions::allow-invalid-utf8 property
---
 c_glib/arrow-glib/compute.cpp | 24 +++-
 c_glib/test/test-cast.rb  | 17 +
 2 files changed, 40 insertions(+), 1 deletion(-)

diff --git a/c_glib/arrow-glib/compute.cpp b/c_glib/arrow-glib/compute.cpp
index 2039eea..a9f6721 100644
--- a/c_glib/arrow-glib/compute.cpp
+++ b/c_glib/arrow-glib/compute.cpp
@@ -42,7 +42,8 @@ enum {
   PROP_0,
   PROP_ALLOW_INT_OVERFLOW,
   PROP_ALLOW_TIME_TRUNCATE,
-  PROP_ALLOW_FLOAT_TRUNCATE
+  PROP_ALLOW_FLOAT_TRUNCATE,
+  PROP_ALLOW_INVALID_UTF8,
 };
 
 G_DEFINE_TYPE_WITH_PRIVATE(GArrowCastOptions,
@@ -72,6 +73,9 @@ garrow_cast_options_set_property(GObject *object,
   case PROP_ALLOW_FLOAT_TRUNCATE:
 priv->options.allow_float_truncate = g_value_get_boolean(value);
 break;
+  case PROP_ALLOW_INVALID_UTF8:
+priv->options.allow_invalid_utf8 = g_value_get_boolean(value);
+break;
   default:
 G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec);
 break;
@@ -96,6 +100,9 @@ garrow_cast_options_get_property(GObject *object,
   case PROP_ALLOW_FLOAT_TRUNCATE:
 g_value_set_boolean(value, priv->options.allow_float_truncate);
 break;
+  case PROP_ALLOW_INVALID_UTF8:
+g_value_set_boolean(value, priv->options.allow_invalid_utf8);
+break;
   default:
 G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec);
 break;
@@ -157,6 +164,20 @@ garrow_cast_options_class_init(GArrowCastOptionsClass 
*klass)
   FALSE,
   static_cast(G_PARAM_READWRITE));
   g_object_class_install_property(gobject_class, PROP_ALLOW_FLOAT_TRUNCATE, 
spec);
+
+  /**
+   * GArrowCastOptions:allow-invalid-utf8:
+   *
+   * Whether invalid UTF-8 string value is allowed or not.
+   *
+   * Since: 0.13.0
+   */
+  spec = g_param_spec_boolean("allow-invalid-utf8",
+  "Allow invalid UTF-8",
+  "Whether invalid UTF-8 string value is allowed 
or not",
+  FALSE,
+  static_cast(G_PARAM_READWRITE));
+  g_object_class_install_property(gobject_class, PROP_ALLOW_INVALID_UTF8, 
spec);
 }
 
 /**
@@ -183,6 +204,7 @@ garrow_cast_options_new_raw(arrow::compute::CastOptions 
*arrow_cast_options)
  "allow-int-overflow", arrow_cast_options->allow_int_overflow,
  "allow-time-truncate", 
arrow_cast_options->allow_time_truncate,
  "allow-float-truncate", 
arrow_cast_options->allow_float_truncate,
+ "allow-invalid-utf8", arrow_cast_options->allow_invalid_utf8,
  NULL);
   return GARROW_CAST_OPTIONS(cast_options);
 }
diff --git a/c_glib/test/test-cast.rb b/c_glib/test/test-cast.rb
index 2512e05..f9d406c 100644
--- a/c_glib/test/test-cast.rb
+++ b/c_glib/test/test-cast.rb
@@ -82,4 +82,21 @@ class TestCast < Test::Unit::TestCase
build_float_array([1.1]).cast(int8_data_type, options))
 end
   end
+
+  sub_test_case("allow-invalid-utf8") do
+def test_default
+  require_gi(1, 42, 0)
+  assert_raise(Arrow::Error::Invalid) do
+build_binary_array(["\xff"]).cast(Arrow::StringDataType.new)
+  end
+end
+
+def test_true
+  options = Arrow::CastOptions.new
+  options.allow_invalid_utf8 = true
+  string_data_type = Arrow::StringDataType.new
+  assert_equal(build_string_array(["\xff"]),
+   build_binary_array(["\xff"]).cast(string_data_type, 
options))
+end
+  end
 end

[arrow] branch master updated: ARROW-4835: [GLib] Add boolean operations

2019-03-12 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new c1debde  ARROW-4835: [GLib] Add boolean operations
c1debde is described below

commit c1debdea66f6b4ea877f8e4053c07996504419d0
Author: Kouhei Sutou 
AuthorDate: Wed Mar 13 09:06:51 2019 +0900

ARROW-4835: [GLib] Add boolean operations

Author: Kouhei Sutou 

Closes #3873 from kou/glib-boolean and squashes the following commits:

16923a14  Fix lint
792b9eb2   Add boolean operations
---
 c_glib/arrow-glib/basic-array.cpp | 137 ++
 c_glib/arrow-glib/basic-array.h   |  19 ++
 c_glib/test/test-boolean-array.rb |  26 
 cpp/src/arrow/compute/api.h   |   5 +-
 4 files changed, 185 insertions(+), 2 deletions(-)

diff --git a/c_glib/arrow-glib/basic-array.cpp 
b/c_glib/arrow-glib/basic-array.cpp
index 9aebd9c..7409945 100644
--- a/c_glib/arrow-glib/basic-array.cpp
+++ b/c_glib/arrow-glib/basic-array.cpp
@@ -787,6 +787,143 @@ garrow_boolean_array_get_values(GArrowBooleanArray *array,
   return values;
 }
 
+/**
+ * garrow_boolean_array_invert:
+ * @array: A #GArrowBooleanArray.
+ * @error: (nullable): Return location for a #GError or %NULL.
+ *
+ * Returns: (transfer full): The element-wise inverted boolean array.
+ *
+ *   It should be freed with g_object_unref() when no longer needed.
+ *
+ * Since: 0.13.0
+ */
+GArrowBooleanArray *
+garrow_boolean_array_invert(GArrowBooleanArray *array,
+GError **error)
+{
+  auto arrow_array = garrow_array_get_raw(GARROW_ARRAY(array));
+  auto datum = arrow::compute::Datum(arrow_array);
+  auto memory_pool = arrow::default_memory_pool();
+  arrow::compute::FunctionContext context(memory_pool);
+  arrow::compute::Datum inverted_datum;
+  auto status = arrow::compute::Invert(, datum, _datum);
+  if (garrow_error_check(error, status, "[boolean-array][invert]")) {
+auto arrow_inverted_array = inverted_datum.make_array();
+return GARROW_BOOLEAN_ARRAY(garrow_array_new_raw(_inverted_array));
+  } else {
+return NULL;
+  }
+}
+
+/**
+ * garrow_boolean_array_and:
+ * @left: A left hand side #GArrowBooleanArray.
+ * @right: A right hand side #GArrowBooleanArray.
+ * @error: (nullable): Return location for a #GError or %NULL.
+ *
+ * Returns: (transfer full): The element-wise AND operated boolean array.
+ *
+ *   It should be freed with g_object_unref() when no longer needed.
+ *
+ * Since: 0.13.0
+ */
+GArrowBooleanArray *
+garrow_boolean_array_and(GArrowBooleanArray *left,
+ GArrowBooleanArray *right,
+ GError **error)
+{
+  auto arrow_left = garrow_array_get_raw(GARROW_ARRAY(left));
+  auto left_datum = arrow::compute::Datum(arrow_left);
+  auto arrow_right = garrow_array_get_raw(GARROW_ARRAY(right));
+  auto right_datum = arrow::compute::Datum(arrow_right);
+  auto memory_pool = arrow::default_memory_pool();
+  arrow::compute::FunctionContext context(memory_pool);
+  arrow::compute::Datum operated_datum;
+  auto status = arrow::compute::And(,
+left_datum,
+right_datum,
+_datum);
+  if (garrow_error_check(error, status, "[boolean-array][and]")) {
+auto arrow_operated_array = operated_datum.make_array();
+return GARROW_BOOLEAN_ARRAY(garrow_array_new_raw(_operated_array));
+  } else {
+return NULL;
+  }
+}
+
+/**
+ * garrow_boolean_array_or:
+ * @left: A left hand side #GArrowBooleanArray.
+ * @right: A right hand side #GArrowBooleanArray.
+ * @error: (nullable): Return location for a #GError or %NULL.
+ *
+ * Returns: (transfer full): The element-wise OR operated boolean array.
+ *
+ *   It should be freed with g_object_unref() when no longer needed.
+ *
+ * Since: 0.13.0
+ */
+GArrowBooleanArray *
+garrow_boolean_array_or(GArrowBooleanArray *left,
+GArrowBooleanArray *right,
+GError **error)
+{
+  auto arrow_left = garrow_array_get_raw(GARROW_ARRAY(left));
+  auto left_datum = arrow::compute::Datum(arrow_left);
+  auto arrow_right = garrow_array_get_raw(GARROW_ARRAY(right));
+  auto right_datum = arrow::compute::Datum(arrow_right);
+  auto memory_pool = arrow::default_memory_pool();
+  arrow::compute::FunctionContext context(memory_pool);
+  arrow::compute::Datum operated_datum;
+  auto status = arrow::compute::Or(,
+   left_datum,
+   right_datum,
+   _datum);
+  if (garrow_error_check(error, status, "[boolean-array][or]")) {
+auto arrow_operated_array = operated_datum.make_array();
+return GARROW_BOOLEAN_ARRAY(garrow_array_new_raw(_operated_array))

[arrow] branch master updated: ARROW-4813: [Ruby] Add tests for == and !=

2019-03-09 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 0ad791e  ARROW-4813: [Ruby] Add tests for == and !=
0ad791e is described below

commit 0ad791e64759e4bbc69aad195b7d31aeb92f0cf9
Author: Kouhei Sutou 
AuthorDate: Sun Mar 10 10:09:22 2019 +0900

ARROW-4813: [Ruby] Add tests for == and !=

Author: Kouhei Sutou 

Closes #3853 from kou/ruby-equal-test and squashes the following commits:

7b8c52bc   Add tests for == and !=
---
 ruby/red-arrow/red-arrow.gemspec   |  2 +-
 ruby/red-arrow/test/test-array.rb  | 14 ++
 .../test/{test-array.rb => test-buffer.rb} | 37 +--
 ruby/red-arrow/test/test-chunked-array.rb  | 22 +
 ruby/red-arrow/test/test-column.rb | 24 ++
 ruby/red-arrow/test/test-data-type.rb  | 20 
 .../test/{test-array.rb => test-decimal128.rb} | 45 +-
 ruby/red-arrow/test/test-field.rb  | 20 
 ruby/red-arrow/test/test-record-batch.rb   | 14 ++
 ruby/red-arrow/test/test-schema.rb | 14 ++
 ruby/red-arrow/test/test-table.rb  | 14 ++
 .../test/{test-array.rb => test-tensor.rb} | 53 --
 12 files changed, 204 insertions(+), 75 deletions(-)

diff --git a/ruby/red-arrow/red-arrow.gemspec b/ruby/red-arrow/red-arrow.gemspec
index 2d417f0..121f567 100644
--- a/ruby/red-arrow/red-arrow.gemspec
+++ b/ruby/red-arrow/red-arrow.gemspec
@@ -45,7 +45,7 @@ Gem::Specification.new do |spec|
   spec.test_files += Dir.glob("test/**/*")
   spec.extensions = ["dependency-check/Rakefile"]
 
-  spec.add_runtime_dependency("gobject-introspection", ">= 3.3.1")
+  spec.add_runtime_dependency("gobject-introspection", ">= 3.3.5")
   spec.add_runtime_dependency("pkg-config")
   spec.add_runtime_dependency("native-package-installer")
 
diff --git a/ruby/red-arrow/test/test-array.rb 
b/ruby/red-arrow/test/test-array.rb
index 3dd7635..793a262 100644
--- a/ruby/red-arrow/test/test-array.rb
+++ b/ruby/red-arrow/test/test-array.rb
@@ -49,5 +49,19 @@ class ArrayTest < Test::Unit::TestCase
  @array[-1])
   end
 end
+
+sub_test_case("#==") do
+  test("Arrow::Array") do
+assert do
+  @array == @array
+end
+  end
+
+  test("not Arrow::Array") do
+assert do
+  not (@array == 29)
+end
+  end
+end
   end
 end
diff --git a/ruby/red-arrow/test/test-array.rb 
b/ruby/red-arrow/test/test-buffer.rb
similarity index 54%
copy from ruby/red-arrow/test/test-array.rb
copy to ruby/red-arrow/test/test-buffer.rb
index 3dd7635..64f1907 100644
--- a/ruby/red-arrow/test/test-array.rb
+++ b/ruby/red-arrow/test/test-buffer.rb
@@ -15,38 +15,23 @@
 # specific language governing permissions and limitations
 # under the License.
 
-class ArrayTest < Test::Unit::TestCase
-  sub_test_case(".new") do
-test("Boolean") do
-  array = Arrow::BooleanArray.new([true, false, true])
-  assert_equal([true, false, true],
-   array.to_a)
-end
-  end
-
+class BufferTest < Test::Unit::TestCase
   sub_test_case("instance methods") do
 def setup
-  @values = [true, false, nil, true]
-  @array = Arrow::BooleanArray.new(@values)
+  @buffer = Arrow::Buffer.new("Hello")
 end
 
-test("#each") do
-  assert_equal(@values, @array.to_a)
-end
-
-sub_test_case("#[]") do
-  test("valid range") do
-assert_equal(@values,
- @array.length.times.collect {|i| @array[i]})
-  end
-
-  test("out of range") do
-assert_nil(@array[@array.length])
+sub_test_case("#==") do
+  test("Arrow::Buffer") do
+assert do
+  @buffer == @buffer
+end
   end
 
-  test("negative index") do
-assert_equal(@values.last,
- @array[-1])
+  test("not Arrow::Buffer") do
+assert do
+  not (@buffer == 29)
+end
   end
 end
   end
diff --git a/ruby/red-arrow/test/test-chunked-array.rb 
b/ruby/red-arrow/test/test-chunked-array.rb
index 2344d80..853922a 100644
--- a/ruby/red-arrow/test/test-chunked-array.rb
+++ b/ruby/red-arrow/test/test-chunked-array.rb
@@ -62,4 +62,26 @@ class ChunkedArrayTest < Test::Unit::TestCase
])
 end
   end
+
+  sub_test_case("#==") do
+def setup
+  arrays = [
+Arrow::BooleanArray.new([true]),
+Arrow::BooleanArray.ne

[arrow] branch master updated: ARROW-4801: [GLib] Suppress Meson warnings

2019-03-09 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 17500f7  ARROW-4801: [GLib] Suppress Meson warnings
17500f7 is described below

commit 17500f781282f21a68ea818d21a9006df3ee0d42
Author: Kouhei Sutou 
AuthorDate: Sun Mar 10 09:32:15 2019 +0900

ARROW-4801: [GLib] Suppress Meson warnings

Messages:

arrow-glib/meson.build:211: DEPRECATION: Library arrow-glib was passed 
to the "libraries" keyword argument of a previous call to generate() method 
instead of first positional argument. Adding arrow-glib to "Requires" field, 
but this is a deprecated behaviour that will change in a future version of 
Meson. Please report the issue if this warning cannot be avoided in your case.
arrow-cuda-glib/meson.build:53: DEPRECATION: Library arrow-cuda-glib 
was passed to the "libraries" keyword argument of a previous call to generate() 
method instead of first positional argument. Adding arrow-cuda-glib to 
"Requires" field, but this is a deprecated behaviour that will change in a 
future version of Meson. Please report the issue if this warning cannot be 
avoided in your case.

Author: Kouhei Sutou 

Closes #3840 from kou/glib-meson-suppress-warning and squashes the 
following commits:

3c0b0bbc   Suppress Meson warnings
---
 c_glib/arrow-cuda-glib/meson.build |  6 +++---
 c_glib/arrow-glib/meson.build  |  6 +++---
 c_glib/gandiva-glib/meson.build|  6 +++---
 c_glib/parquet-glib/meson.build| 12 ++--
 c_glib/plasma-glib/meson.build |  6 +++---
 5 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/c_glib/arrow-cuda-glib/meson.build 
b/c_glib/arrow-cuda-glib/meson.build
index e5b9f47..a29f12b 100644
--- a/c_glib/arrow-cuda-glib/meson.build
+++ b/c_glib/arrow-cuda-glib/meson.build
@@ -50,12 +50,12 @@ arrow_cuda_glib = declare_dependency(link_with: 
libarrow_cuda_glib,
  include_directories: 
base_include_directories,
  dependencies: dependencies)
 
-pkgconfig.generate(filebase: 'arrow-cuda-glib',
+pkgconfig.generate(libarrow_cuda_glib,
+   filebase: 'arrow-cuda-glib',
name: 'Apache Arrow CUDA GLib',
description: 'C API for Apache Arrow CUDA based on GLib',
version: version,
-   requires: ['arrow-glib', 'arrow-cuda'],
-   libraries: [libarrow_cuda_glib])
+   requires: ['arrow-glib', 'arrow-cuda'])
 
 gir_dependencies = [
   declare_dependency(sources: arrow_glib_gir),
diff --git a/c_glib/arrow-glib/meson.build b/c_glib/arrow-glib/meson.build
index 14126be..4e8e1cc 100644
--- a/c_glib/arrow-glib/meson.build
+++ b/c_glib/arrow-glib/meson.build
@@ -208,12 +208,12 @@ arrow_glib = declare_dependency(link_with: libarrow_glib,
 dependencies: dependencies,
 sources: enums_header)
 
-pkgconfig.generate(filebase: meson.project_name(),
+pkgconfig.generate(libarrow_glib,
+   filebase: meson.project_name(),
name: 'Apache Arrow GLib',
description: 'C API for Apache Arrow based on GLib',
version: version,
-   requires: ['gio-2.0', 'arrow'],
-   libraries: [libarrow_glib])
+   requires: ['gio-2.0', 'arrow'])
 if have_arrow_orc
   pkgconfig.generate(filebase: 'arrow-orc-glib',
  name: 'Apache Arrow GLib ORC',
diff --git a/c_glib/gandiva-glib/meson.build b/c_glib/gandiva-glib/meson.build
index 7c288e4..11ef8d6 100644
--- a/c_glib/gandiva-glib/meson.build
+++ b/c_glib/gandiva-glib/meson.build
@@ -58,12 +58,12 @@ gandiva_glib = declare_dependency(link_with: 
libgandiva_glib,
   dependencies: dependencies,
   sources: enums_header)
 
-pkgconfig.generate(filebase: project_name,
+pkgconfig.generate(libgandiva_glib,
+   filebase: project_name,
name: 'Apache Arrow Gandiva GLib',
description: 'C API for Apache Arrow Gandiva based on GLib',
version: version,
-   requires: ['gandiva', 'arrow-glib'],
-   libraries: [libgandiva_glib])
+   requires: ['gandiva', 'arrow-glib'])
 
 gnome.generate_gir(libgandiva_glib,
dependencies: declare_dependency(sources: arrow_glib_gir),
diff --git a/c_glib/parquet-glib/meson.build b/c_glib/parquet-glib/meson.build
index afa6d21..a52b328 100644
--- a/c_glib/parquet-glib/meson.build
+++ b/c_glib/parquet-glib/meson.build
@@ -56,12 +56,12 @@ parquet_glib = declar

[arrow] branch master updated: ARROW-4793: [Ruby] Suppress unused variable warning

2019-03-07 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new dccf7f7  ARROW-4793: [Ruby] Suppress unused variable warning
dccf7f7 is described below

commit dccf7f7bb53482c759b48b0449abd32b92d7bbfc
Author: Kouhei Sutou 
AuthorDate: Thu Mar 7 20:14:36 2019 +0900

ARROW-4793: [Ruby] Suppress unused variable warning

Message:

ruby/red-arrow/test/test-bigdecimal.rb:20: warning: assigned but unused 
variable - arrow_decimal

Author: Kouhei Sutou 

Closes #3828 from kou/ruby-suppress-warning and squashes the following 
commits:

2f6c7e26   Suppress unused variable warning
---
 ruby/red-arrow/test/test-bigdecimal.rb | 1 -
 1 file changed, 1 deletion(-)

diff --git a/ruby/red-arrow/test/test-bigdecimal.rb 
b/ruby/red-arrow/test/test-bigdecimal.rb
index 3874a38..e25881c 100644
--- a/ruby/red-arrow/test/test-bigdecimal.rb
+++ b/ruby/red-arrow/test/test-bigdecimal.rb
@@ -17,7 +17,6 @@
 
 class BigDecimalTest < Test::Unit::TestCase
   test("#to_arrow") do
-arrow_decimal = BigDecimal("3.14").to_arrow
 assert_equal(Arrow::Decimal128.new("3.14"),
  BigDecimal("3.14").to_arrow)
   end

[arrow] branch master updated: ARROW-4593: [Ruby] Arrow::Array#[out_of_range] returns nil

2019-02-16 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 0c340b4  ARROW-4593: [Ruby] Arrow::Array#[out_of_range] returns nil
0c340b4 is described below

commit 0c340b4a4a1bcd71c3c1a97415d6c06ceaf43c2d
Author: Kouhei Sutou 
AuthorDate: Sun Feb 17 10:03:52 2019 +0900

ARROW-4593: [Ruby] Arrow::Array#[out_of_range] returns nil

Author: Kouhei Sutou 

Closes #3666 from kou/ruby-array-ref-out-of-range and squashes the 
following commits:

f64df5b5   Array# returns nil
---
 ruby/red-arrow/lib/arrow/array.rb | 10 ++
 ruby/red-arrow/test/test-array.rb | 33 -
 2 files changed, 34 insertions(+), 9 deletions(-)

diff --git a/ruby/red-arrow/lib/arrow/array.rb 
b/ruby/red-arrow/lib/arrow/array.rb
index 359e70e..f60025a 100644
--- a/ruby/red-arrow/lib/arrow/array.rb
+++ b/ruby/red-arrow/lib/arrow/array.rb
@@ -35,8 +35,18 @@ module Arrow
   end
 end
 
+# @param i [Integer]
+#   The index of the value to be gotten.
+#
+#   You can specify negative index like for `::Array#[]`.
+#
+# @return [Object, nil]
+#   The `i`-th value.
+#
+#   `nil` for NULL value or out of range `i`.
 def [](i)
   i += length if i < 0
+  return nil if i < 0 or i >= length
   if null?(i)
 nil
   else
diff --git a/ruby/red-arrow/test/test-array.rb 
b/ruby/red-arrow/test/test-array.rb
index 31e6eaf..3dd7635 100644
--- a/ruby/red-arrow/test/test-array.rb
+++ b/ruby/red-arrow/test/test-array.rb
@@ -24,15 +24,30 @@ class ArrayTest < Test::Unit::TestCase
 end
   end
 
-  test("#each") do
-array = Arrow::BooleanArray.new([true, false, nil, true])
-assert_equal([true, false, nil, true],
- array.to_a)
-  end
+  sub_test_case("instance methods") do
+def setup
+  @values = [true, false, nil, true]
+  @array = Arrow::BooleanArray.new(@values)
+end
+
+test("#each") do
+  assert_equal(@values, @array.to_a)
+end
 
-  test("#[]") do
-array = Arrow::BooleanArray.new([true, false, nil, true])
-assert_equal([true, false, nil, true],
- [array[0], array[1], array[2], array[3]])
+sub_test_case("#[]") do
+  test("valid range") do
+assert_equal(@values,
+ @array.length.times.collect {|i| @array[i]})
+  end
+
+  test("out of range") do
+assert_nil(@array[@array.length])
+  end
+
+  test("negative index") do
+assert_equal(@values.last,
+ @array[-1])
+  end
+end
   end
 end

[arrow] branch master updated: ARROW-4227: [GLib] Fix wrong data type in field of composite data type

2019-01-10 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new bf34291  ARROW-4227: [GLib] Fix wrong data type in field of composite 
data type
bf34291 is described below

commit bf34291b93c748f9dc63a0d89cc1cf857a28630c
Author: Kouhei Sutou 
AuthorDate: Thu Jan 10 18:54:20 2019 +0900

ARROW-4227: [GLib] Fix wrong data type in field of composite data type

Author: Kouhei Sutou 

Closes #3363 from kou/glib-fix-wrong-data-type and squashes the following 
commits:

1f274e51  Use garrow_field_new_raw(..., nullpter)
20460a9f   Fix wrong data type in field of composite data type
---
 c_glib/arrow-glib/composite-data-type.cpp  | 14 +--
 c_glib/arrow-glib/field.cpp|  9 +++
 c_glib/arrow-glib/schema.cpp   | 16 +++-
 c_glib/test/test-dense-union-data-type.rb  | 33 ++---
 c_glib/test/test-list-data-type.rb | 25 ---
 c_glib/test/test-sparse-union-data-type.rb | 33 ++---
 c_glib/test/test-struct-data-type.rb   | 39 +++---
 7 files changed, 127 insertions(+), 42 deletions(-)

diff --git a/c_glib/arrow-glib/composite-data-type.cpp 
b/c_glib/arrow-glib/composite-data-type.cpp
index 8046d2e..5ddc1c3 100644
--- a/c_glib/arrow-glib/composite-data-type.cpp
+++ b/c_glib/arrow-glib/composite-data-type.cpp
@@ -98,7 +98,7 @@ garrow_list_data_type_get_value_field(GArrowListDataType 
*list_data_type)
 static_cast(arrow_data_type.get());
 
   auto arrow_field = arrow_list_data_type->value_field();
-  return garrow_field_new_raw(_field, data_type);
+  return garrow_field_new_raw(_field, nullptr);
 }
 
 
@@ -172,8 +172,7 @@ garrow_struct_data_type_get_fields(GArrowStructDataType 
*struct_data_type)
 
   GList *fields = NULL;
   for (auto arrow_field : arrow_fields) {
-fields = g_list_prepend(fields,
-garrow_field_new_raw(_field, data_type));
+fields = g_list_prepend(fields, garrow_field_new_raw(_field, 
nullptr));
   }
   return g_list_reverse(fields);
 }
@@ -207,7 +206,7 @@ garrow_struct_data_type_get_field(GArrowStructDataType 
*struct_data_type,
 
   auto arrow_field = arrow_data_type->child(i);
   if (arrow_field) {
-return garrow_field_new_raw(_field, data_type);
+return garrow_field_new_raw(_field, nullptr);
   } else {
 return NULL;
   }
@@ -234,7 +233,7 @@ 
garrow_struct_data_type_get_field_by_name(GArrowStructDataType *struct_data_type
 
   auto arrow_field = arrow_struct_data_type->GetFieldByName(name);
   if (arrow_field) {
-return garrow_field_new_raw(_field, data_type);
+return garrow_field_new_raw(_field, nullptr);
   } else {
 return NULL;
   }
@@ -309,8 +308,7 @@ garrow_union_data_type_get_fields(GArrowUnionDataType 
*union_data_type)
 
   GList *fields = NULL;
   for (auto arrow_field : arrow_fields) {
-fields = g_list_prepend(fields,
-garrow_field_new_raw(_field, data_type));
+fields = g_list_prepend(fields, garrow_field_new_raw(_field, 
nullptr));
   }
   return g_list_reverse(fields);
 }
@@ -344,7 +342,7 @@ garrow_union_data_type_get_field(GArrowUnionDataType 
*union_data_type,
 
   auto arrow_field = arrow_data_type->child(i);
   if (arrow_field) {
-return garrow_field_new_raw(_field, data_type);
+return garrow_field_new_raw(_field, nullptr);
   } else {
 return NULL;
   }
diff --git a/c_glib/arrow-glib/field.cpp b/c_glib/arrow-glib/field.cpp
index d74053a..f7250bc 100644
--- a/c_glib/arrow-glib/field.cpp
+++ b/c_glib/arrow-glib/field.cpp
@@ -243,10 +243,19 @@ GArrowField *
 garrow_field_new_raw(std::shared_ptr *arrow_field,
  GArrowDataType *data_type)
 {
+  bool data_type_need_unref = false;
+  if (!data_type) {
+auto arrow_data_type = (*arrow_field)->type();
+data_type = garrow_data_type_new_raw(_data_type);
+data_type_need_unref = true;
+  }
   auto field = GARROW_FIELD(g_object_new(GARROW_TYPE_FIELD,
  "field", arrow_field,
  "data-type", data_type,
  NULL));
+  if (data_type_need_unref) {
+g_object_unref(data_type);
+  }
   return field;
 }
 
diff --git a/c_glib/arrow-glib/schema.cpp b/c_glib/arrow-glib/schema.cpp
index 6433241..1bbe82f 100644
--- a/c_glib/arrow-glib/schema.cpp
+++ b/c_glib/arrow-glib/schema.cpp
@@ -174,11 +174,7 @@ garrow_schema_get_field(GArrowSchema *schema, guint i)
 {
   const auto arrow_schema = garrow_schema_get_raw(schema);
   auto arrow_field = arrow_schema->field(i);
-  auto arrow_data_type = arrow_field->type();
-  auto data_type = garrow_data_type_new_raw(_data_type);
-  auto field = garrow_field_new_raw(_field, data_type)

[arrow] branch master updated: ARROW-4215: [GLib] Fix typos in documentation

2019-01-09 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new db29723  ARROW-4215: [GLib] Fix typos in documentation
db29723 is described below

commit db29723f661174eefd04077666347a9bbaca5be1
Author: Kouhei Sutou 
AuthorDate: Thu Jan 10 11:49:29 2019 +0900

ARROW-4215: [GLib] Fix typos in documentation

This solves the following warnings:

arrow-glib/basic-data-type.cpp:1070: warning: multi-line since docs 
found
arrow-glib/decimal128.cpp:37: warning: Section decimal is not defined 
in the arrow-glib-sections.txt file.

Author: Kouhei Sutou 

Closes #3361 from kou/glib-fix-document and squashes the following commits:

edd43c8a   Fix typos in documentation
---
 c_glib/arrow-glib/basic-data-type.cpp | 2 +-
 c_glib/arrow-glib/decimal128.cpp  | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/c_glib/arrow-glib/basic-data-type.cpp 
b/c_glib/arrow-glib/basic-data-type.cpp
index 2a59996..861bbaf 100644
--- a/c_glib/arrow-glib/basic-data-type.cpp
+++ b/c_glib/arrow-glib/basic-data-type.cpp
@@ -1065,7 +1065,7 @@ 
garrow_decimal_data_type_class_init(GArrowDecimalDataTypeClass *klass)
  *
  * Since: 0.10.0
  *
- * Deprecate: 0.12.0:
+ * Deprecated: 0.12.0:
  *   Use garrow_decimal128_data_type_new() instead.
  */
 GArrowDecimalDataType *
diff --git a/c_glib/arrow-glib/decimal128.cpp b/c_glib/arrow-glib/decimal128.cpp
index a49dba5..32bdf5f 100644
--- a/c_glib/arrow-glib/decimal128.cpp
+++ b/c_glib/arrow-glib/decimal128.cpp
@@ -27,8 +27,8 @@
 G_BEGIN_DECLS
 
 /**
- * SECTION: decimal
- * @title: Decimal classes
+ * SECTION: decimal128
+ * @title: 128-bit decimal class
  * @include: arrow-glib/arrow-glib.h
  *
  * #GArrowDecimal128 is a 128-bit decimal class.

[arrow] branch master updated: ARROW-4184: [Ruby] Add Arrow::RecordBatch#to_table

2019-01-08 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new a3aed3b  ARROW-4184: [Ruby] Add Arrow::RecordBatch#to_table
a3aed3b is described below

commit a3aed3b60bd61c55d7402c4484e480f1998b99f1
Author: Kouhei Sutou 
AuthorDate: Wed Jan 9 09:17:46 2019 +0900

ARROW-4184: [Ruby] Add Arrow::RecordBatch#to_table

Author: Kouhei Sutou 

Closes #3339 from kou/ruby-record-batch-to-table and squashes the following 
commits:

a6fab35f  Require gobject-introspection gem 3.3.1 or later
4a1f3564   Add Arrow::RecordBatch#to_table
---
 ruby/red-arrow/lib/arrow/record-batch.rb |  9 +
 ruby/red-arrow/red-arrow.gemspec |  2 +-
 ruby/red-arrow/test/test-record-batch.rb | 23 ++-
 3 files changed, 24 insertions(+), 10 deletions(-)

diff --git a/ruby/red-arrow/lib/arrow/record-batch.rb 
b/ruby/red-arrow/lib/arrow/record-batch.rb
index f5f8ea2..6d9c35b 100644
--- a/ruby/red-arrow/lib/arrow/record-batch.rb
+++ b/ruby/red-arrow/lib/arrow/record-batch.rb
@@ -29,6 +29,15 @@ module Arrow
   @columns ||= columns_raw
 end
 
+# Converts the record batch to {Arrow::Table}.
+#
+# @return [Arrow::Table]
+#
+# @since 0.12.0
+def to_table
+  Table.new(schema, [self])
+end
+
 def respond_to_missing?(name, include_private)
   return true if find_column(name)
   super
diff --git a/ruby/red-arrow/red-arrow.gemspec b/ruby/red-arrow/red-arrow.gemspec
index 8e79c75..2d417f0 100644
--- a/ruby/red-arrow/red-arrow.gemspec
+++ b/ruby/red-arrow/red-arrow.gemspec
@@ -45,7 +45,7 @@ Gem::Specification.new do |spec|
   spec.test_files += Dir.glob("test/**/*")
   spec.extensions = ["dependency-check/Rakefile"]
 
-  spec.add_runtime_dependency("gobject-introspection", ">= 3.1.1")
+  spec.add_runtime_dependency("gobject-introspection", ">= 3.3.1")
   spec.add_runtime_dependency("pkg-config")
   spec.add_runtime_dependency("native-package-installer")
 
diff --git a/ruby/red-arrow/test/test-record-batch.rb 
b/ruby/red-arrow/test/test-record-batch.rb
index 994b16d..4dac085 100644
--- a/ruby/red-arrow/test/test-record-batch.rb
+++ b/ruby/red-arrow/test/test-record-batch.rb
@@ -16,16 +16,16 @@
 # under the License.
 
 class RecordBatchTest < Test::Unit::TestCase
-  sub_test_case(".each") do
-setup do
-  fields = [
-Arrow::Field.new("count", :uint32),
-  ]
-  @schema = Arrow::Schema.new(fields)
-  @counts = Arrow::UInt32Array.new([1, 2, 4, 8])
-  @record_batch = Arrow::RecordBatch.new(@schema, @counts.length, 
[@counts])
-end
+  setup do
+fields = [
+  Arrow::Field.new("count", :uint32),
+]
+@schema = Arrow::Schema.new(fields)
+@counts = Arrow::UInt32Array.new([1, 2, 4, 8])
+@record_batch = Arrow::RecordBatch.new(@schema, @counts.length, [@counts])
+  end
 
+  sub_test_case(".each") do
 test("default") do
   records = []
   @record_batch.each do |record|
@@ -54,4 +54,9 @@ class RecordBatchTest < Test::Unit::TestCase
records.collect {|record, i| [record.index, i]})
 end
   end
+
+  test("#to_table") do
+assert_equal(Arrow::Table.new(@schema, [@counts]),
+ @record_batch.to_table)
+  end
 end

[arrow] branch master updated: ARROW-4183: [Ruby] Add Arrow::Struct as an element of Arrow::StructArray

2019-01-08 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 8704f8b  ARROW-4183: [Ruby] Add Arrow::Struct as an element of 
Arrow::StructArray
8704f8b is described below

commit 8704f8bd98f1edcf1f9ecc51d6fb3b4b5b4ecb88
Author: Kouhei Sutou 
AuthorDate: Tue Jan 8 22:32:13 2019 +0900

ARROW-4183: [Ruby] Add Arrow::Struct as an element of Arrow::StructArray

Returning Arrow::Array by Arrow::StructArray#[] is deprecated.  It'll
return Arrow::Struct in the next release. It's for consistency. All
Arrow::Array#[] implementations should return an element.

Author: Kouhei Sutou 

Closes #3338 from kou/ruby-struct and squashes the following commits:

a0561954   Add Arrow::Struct as an element of 
Arrow::StructArray
---
 ruby/red-arrow/lib/arrow/struct-array-builder.rb |  9 ++-
 ruby/red-arrow/lib/arrow/struct-array.rb | 34 ++
 ruby/red-arrow/lib/arrow/struct.rb   | 68 
 ruby/red-arrow/test/test-struct-array-builder.rb | 47 +-
 ruby/red-arrow/test/test-struct-array.rb | 58 -
 ruby/red-arrow/test/test-struct.rb   | 81 
 6 files changed, 263 insertions(+), 34 deletions(-)

diff --git a/ruby/red-arrow/lib/arrow/struct-array-builder.rb 
b/ruby/red-arrow/lib/arrow/struct-array-builder.rb
index 883ce84..52f75aa 100644
--- a/ruby/red-arrow/lib/arrow/struct-array-builder.rb
+++ b/ruby/red-arrow/lib/arrow/struct-array-builder.rb
@@ -73,13 +73,20 @@ module Arrow
   value.each_with_index do |sub_value, i|
 self[i].append_value(sub_value)
   end
+when Arrow::Struct
+  append_value_raw
+  value.values.each_with_index do |sub_value, i|
+self[i].append_value(sub_value)
+  end
 when Hash
   append_value_raw
   value.each do |name, sub_value|
 self[name].append_value(sub_value)
   end
 else
-  message = "struct value must be nil, Array or Hash: #{value.inspect}"
+  message =
+"struct value must be nil, Array, " +
+"Arrow::Struct or Hash: #{value.inspect}"
   raise ArgumentError, message
 end
   else
diff --git a/ruby/red-arrow/lib/arrow/struct-array.rb 
b/ruby/red-arrow/lib/arrow/struct-array.rb
index 4f9834c..e55a507 100644
--- a/ruby/red-arrow/lib/arrow/struct-array.rb
+++ b/ruby/red-arrow/lib/arrow/struct-array.rb
@@ -15,10 +15,44 @@
 # specific language governing permissions and limitations
 # under the License.
 
+require "arrow/struct"
+
 module Arrow
   class StructArray
 def [](i)
+  warn("Use #{self.class}\#find_field instead. " +
+   "This will returns Arrow::Struct instead of Arrow::Array " +
+   "since 0.13.0.")
   get_field(i)
 end
+
+def get_value(i)
+  Struct.new(self, i)
+end
+
+def find_field(index_or_name)
+  case index_or_name
+  when String, Symbol
+name = index_or_name
+(@name_to_field ||= build_name_to_field)[name.to_s]
+  else
+index = index_or_name
+cached_fields[index]
+  end
+end
+
+private
+def cached_fields
+  @fields ||= fields
+end
+
+def build_name_to_field
+  name_to_field = {}
+  field_arrays = cached_fields
+  value_data_type.fields.each_with_index do |field, i|
+name_to_field[field.name] = field_arrays[i]
+  end
+  name_to_field
+end
   end
 end
diff --git a/ruby/red-arrow/lib/arrow/struct.rb 
b/ruby/red-arrow/lib/arrow/struct.rb
new file mode 100644
index 000..4ae12b8
--- /dev/null
+++ b/ruby/red-arrow/lib/arrow/struct.rb
@@ -0,0 +1,68 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+module Arrow
+  class Struct
+attr_accessor :index
+def initialize(array, index)
+  @array = array
+  @index = index
+end
+
+def [](field_name_or_field_index)
+  field = @array

[arrow] branch master updated: ARROW-4174: [Ruby] Add support for building composite array from raw Ruby objects

2019-01-07 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 84e10b6  ARROW-4174: [Ruby] Add support for building composite array 
from raw Ruby objects
84e10b6 is described below

commit 84e10b69a8043f507eabc7b3f224a265baa33a1a
Author: Kouhei Sutou 
AuthorDate: Mon Jan 7 19:58:39 2019 +0900

ARROW-4174: [Ruby] Add support for building composite array from raw Ruby 
objects

Author: Kouhei Sutou 

Closes #3327 from kou/ruby-array-builder and squashes the following commits:

20e5874c  Add support for old GObject Introspection
36b993ba   Add support for building composite array from raw 
Ruby objects
---
 c_glib/arrow-glib/array-builder.cpp|  21 +++
 c_glib/arrow-glib/array-builder.h  |   3 +
 c_glib/arrow-glib/decimal128.cpp   |  18 +++
 c_glib/arrow-glib/decimal128.h |   4 +
 ruby/red-arrow/lib/arrow/array.rb  |   8 +-
 .../{array.rb => decimal128-array-builder.rb}  |  71 +-
 ruby/red-arrow/lib/arrow/field.rb  |   2 +-
 ruby/red-arrow/lib/arrow/list-array-builder.rb |  86 
 ruby/red-arrow/lib/arrow/loader.rb |   5 +-
 ruby/red-arrow/lib/arrow/struct-array-builder.rb   | 129 ++
 .../test/test-decimal128-array-builder.rb  |  95 ++
 ...st-struct-array.rb => test-decimal128-array.rb} |  36 ++---
 ruby/red-arrow/test/test-list-array-builder.rb |  62 +
 .../{test-struct-array.rb => test-list-array.rb}   |  30 ++---
 ruby/red-arrow/test/test-struct-array-builder.rb   | 145 +
 ruby/red-arrow/test/test-struct-array.rb   |  21 +++
 16 files changed, 661 insertions(+), 75 deletions(-)

diff --git a/c_glib/arrow-glib/array-builder.cpp 
b/c_glib/arrow-glib/array-builder.cpp
index 5f2d411..095c68d 100644
--- a/c_glib/arrow-glib/array-builder.cpp
+++ b/c_glib/arrow-glib/array-builder.cpp
@@ -3863,6 +3863,27 @@ 
garrow_decimal128_array_builder_append_value(GArrowDecimal128ArrayBuilder *build
  "[decimal128-array-builder][append-value]");
 }
 
+/**
+ * garrow_decimal128_array_builder_append_null:
+ * @builder: A #GArrowDecimal128ArrayBuilder.
+ * @error: (nullable): Return location for a #GError or %NULL.
+ *
+ * Returns: %TRUE on success, %FALSE if there was an error.
+ *
+ * It appends a new NULL element.
+ *
+ * Since: 0.12.0
+ */
+gboolean
+garrow_decimal128_array_builder_append_null(GArrowDecimal128ArrayBuilder 
*builder,
+GError **error)
+{
+  return garrow_array_builder_append_null
+(GARROW_ARRAY_BUILDER(builder),
+ error,
+ "[decimal128-array-builder][append-null]");
+}
+
 G_END_DECLS
 
 GArrowArrayBuilder *
diff --git a/c_glib/arrow-glib/array-builder.h 
b/c_glib/arrow-glib/array-builder.h
index b2ad6f4..bc0a994 100644
--- a/c_glib/arrow-glib/array-builder.h
+++ b/c_glib/arrow-glib/array-builder.h
@@ -1486,5 +1486,8 @@ GARROW_AVAILABLE_IN_0_12
 gboolean 
garrow_decimal128_array_builder_append_value(GArrowDecimal128ArrayBuilder 
*builder,
   GArrowDecimal128 *value,
   GError **error);
+GARROW_AVAILABLE_IN_0_12
+gboolean 
garrow_decimal128_array_builder_append_null(GArrowDecimal128ArrayBuilder 
*builder,
+ GError **error);
 
 G_END_DECLS
diff --git a/c_glib/arrow-glib/decimal128.cpp b/c_glib/arrow-glib/decimal128.cpp
index e30eb7e..d87a501 100644
--- a/c_glib/arrow-glib/decimal128.cpp
+++ b/c_glib/arrow-glib/decimal128.cpp
@@ -137,6 +137,24 @@ garrow_decimal128_new_integer(const gint64 data)
 }
 
 /**
+ * garrow_decimal128_equal:
+ * @decimal: A #GArrowDecimal128.
+ * @other_decimal: A #GArrowDecimal128 to be compared.
+ *
+ * Returns: %TRUE if both of them is the same value, %FALSE otherwise.
+ *
+ * Since: 0.12.0
+ */
+gboolean
+garrow_decimal128_equal(GArrowDecimal128 *decimal,
+GArrowDecimal128 *other_decimal)
+{
+  const auto arrow_decimal = garrow_decimal128_get_raw(decimal);
+  const auto arrow_other_decimal = garrow_decimal128_get_raw(other_decimal);
+  return *arrow_decimal == *arrow_other_decimal;
+}
+
+/**
  * garrow_decimal128_to_string_scale:
  * @decimal: A #GArrowDecimal128.
  * @scale: The scale of the decimal.
diff --git a/c_glib/arrow-glib/decimal128.h b/c_glib/arrow-glib/decimal128.h
index 918cf3d..e8fa599 100644
--- a/c_glib/arrow-glib/decimal128.h
+++ b/c_glib/arrow-glib/decimal128.h
@@ -20,6 +20,7 @@
 #pragma once
 
 #include 
+#include 
 
 G_BEGIN_DECLS
 
@@ -37,6 +38,9 @@ struct _GArrowDecimal128Class
 
 GArrowDecimal128 *garrow_decimal128_new_string(const gchar *data);
 GArrowDecimal128 *garrow

[arrow] branch master updated: ARROW-4166: [Ruby] Add support for saving to and loading from buffer

2019-01-06 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new b95628f  ARROW-4166: [Ruby] Add support for saving to and loading from 
buffer
b95628f is described below

commit b95628f2980fd800efe73ab0e4778dd209f7596c
Author: Kouhei Sutou 
AuthorDate: Mon Jan 7 08:54:59 2019 +0900

ARROW-4166: [Ruby] Add support for saving to and loading from buffer

Author: Kouhei Sutou 

Closes #3320 from kou/ruby-table-io-buffer and squashes the following 
commits:

7025e765   Add support for saving to and loading from buffer
---
 ruby/red-arrow/lib/arrow/table-loader.rb   |  46 ---
 ruby/red-arrow/lib/arrow/table-saver.rb|  66 +-
 ruby/red-arrow/test/test-table.rb  | 139 ++---
 .../lib/parquet/arrow-table-loadable.rb|   7 +-
 .../red-parquet/lib/parquet/arrow-table-savable.rb |   6 +-
 ruby/red-parquet/test/test-arrow-table.rb  |   8 +-
 6 files changed, 177 insertions(+), 95 deletions(-)

diff --git a/ruby/red-arrow/lib/arrow/table-loader.rb 
b/ruby/red-arrow/lib/arrow/table-loader.rb
index a6ce9a1..9bfd410 100644
--- a/ruby/red-arrow/lib/arrow/table-loader.rb
+++ b/ruby/red-arrow/lib/arrow/table-loader.rb
@@ -18,14 +18,14 @@
 module Arrow
   class TableLoader
 class << self
-  def load(path, options={})
-new(path, options).load
+  def load(output, options={})
+new(output, options).load
   end
 end
 
-def initialize(path, options={})
-  path = path.to_path if path.respond_to?(:to_path)
-  @path = path
+def initialize(output, options={})
+  output = output.to_path if output.respond_to?(:to_path)
+  @output = output
   @options = options
   fill_options
 end
@@ -50,7 +50,7 @@ module Arrow
 __send__(custom_load_method)
   else
 # For backward compatibility.
-__send__(custom_load_method, @path)
+__send__(custom_load_method, @output)
   end
 end
 
@@ -60,11 +60,15 @@ module Arrow
 return
   end
 
-  extension = PathExtension.new(@path)
-  info = extension.extract
+  if @output.is_a?(Buffer)
+info = {}
+  else
+extension = PathExtension.new(@output)
+info = extension.extract
+  end
   format = info[:format]
   @options = @options.dup
-  if respond_to?("load_as_#{format}", true)
+  if format and respond_to?("load_as_#{format}", true)
 @options[:format] ||= format.to_sym
   else
 @options[:format] ||= :arrow
@@ -74,6 +78,14 @@ module Arrow
   end
 end
 
+def open_input_stream
+  if @output.is_a?(Buffer)
+BufferInputStream.new(@output)
+  else
+MemoryMappedInputStream.new(@output)
+  end
+end
+
 def load_raw(input, reader)
   schema = reader.schema
   chunked_arrays = []
@@ -100,7 +112,7 @@ module Arrow
 RecordBatchStreamReader,
   ]
   reader_class_candidates.each do |reader_class_candidate|
-input = MemoryMappedInputStream.new(@path)
+input = open_input_stream
 begin
   reader = reader_class_candidate.new(input)
 rescue Arrow::Error
@@ -114,20 +126,20 @@ module Arrow
 end
 
 def load_as_batch
-  input = MemoryMappedInputStream.new(@path)
+  input = open_input_stream
   reader = RecordBatchFileReader.new(input)
   load_raw(input, reader)
 end
 
 def load_as_stream
-  input = MemoryMappedInputStream.new(@path)
+  input = open_input_stream
   reader = RecordBatchStreamReader.new(input)
   load_raw(input, reader)
 end
 
 if Arrow.const_defined?(:ORCFileReader)
   def load_as_orc
-input = MemoryMappedInputStream.new(@path)
+input = open_input_stream
 reader = ORCFileReader.new(input)
 field_indexes = @options[:field_indexes]
 reader.set_field_indexes(field_indexes) if field_indexes
@@ -140,11 +152,15 @@ module Arrow
 def load_as_csv
   options = @options.dup
   options.delete(:format)
-  CSVLoader.load(Pathname.new(@path), options)
+  if @output.is_a?(Buffer)
+CSVLoader.load(@output.data.to_s, options)
+  else
+CSVLoader.load(Pathname.new(@output), options)
+  end
 end
 
 def load_as_feather
-  input = MemoryMappedInputStream.new(@path)
+  input = open_input_stream
   reader = FeatherFileReader.new(input)
   table = reader.read
   table.instance_variable_set(:@input, input)
diff --git a/ruby/red-arrow/lib/arrow/table-saver.rb 
b/ruby/red-arrow/lib/arrow/table-saver.rb
index 99e6e49..817cc54 100644
--- a/ruby/red-arrow/lib/arrow/table-saver.rb
+++ b/ruby/red-arrow/lib/arrow/table-saver.rb
@@ -18,15 +18,15 @@
 module Arrow
   c

[arrow] branch master updated: ARROW-4161: [GLib] Add PlasmaClientOptions

2019-01-06 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 601498f  ARROW-4161: [GLib] Add PlasmaClientOptions
601498f is described below

commit 601498f7169f2340b393bccba1d0a0e0b65d1562
Author: Kouhei Sutou 
AuthorDate: Sun Jan 6 20:53:13 2019 +0900

ARROW-4161: [GLib] Add PlasmaClientOptions

Author: Kouhei Sutou 

Closes #3315 from kou/glib-plasma-client-new-options and squashes the 
following commits:

73eff12a   Add support for Plasma::ClientOptions
ed52a8ab   Add PlasmaClientOptions
---
 c_glib/plasma-glib/client.cpp  | 137 -
 c_glib/plasma-glib/client.h|  21 
 .../test/plasma/test-plasma-client-options.rb  |  21 ++--
 c_glib/test/plasma/test-plasma-client.rb   |   3 +-
 c_glib/test/plasma/test-plasma-created-object.rb   |   2 +-
 c_glib/test/plasma/test-plasma-referred-object.rb  |   2 +-
 ruby/red-plasma/lib/plasma/client.rb   |  13 +-
 ruby/red-plasma/test/test-plasma-client.rb |  24 +++-
 8 files changed, 201 insertions(+), 22 deletions(-)

diff --git a/c_glib/plasma-glib/client.cpp b/c_glib/plasma-glib/client.cpp
index 9591a0a..2038ea6 100644
--- a/c_glib/plasma-glib/client.cpp
+++ b/c_glib/plasma-glib/client.cpp
@@ -39,6 +39,9 @@ G_BEGIN_DECLS
  * @title: Client related classes
  * @include: plasma-glib/plasma-glib.h
  *
+ * #GPlasmaClientOptions is a class for customizing plasma store
+ * connection.
+ *
  * #GPlasmaClientCreateOptions is a class for customizing object creation.
  *
  * #GPlasmaClient is a class for an interface with a plasma store.
@@ -46,6 +49,131 @@ G_BEGIN_DECLS
  * Since: 0.12.0
  */
 
+typedef struct GPlasmaClientCreatePrivate_ {
+  gint n_retries;
+} GPlasmaClientOptionsPrivate;
+
+enum {
+  PROP_N_RETRIES = 1
+};
+
+G_DEFINE_TYPE_WITH_PRIVATE(GPlasmaClientOptions,
+   gplasma_client_options,
+   G_TYPE_OBJECT)
+
+#define GPLASMA_CLIENT_OPTIONS_GET_PRIVATE(object)  \
+  static_cast(   \
+gplasma_client_options_get_instance_private(\
+  GPLASMA_CLIENT_OPTIONS(object)))
+
+static void
+gplasma_client_options_set_property(GObject *object,
+guint prop_id,
+const GValue *value,
+GParamSpec *pspec)
+{
+  auto priv = GPLASMA_CLIENT_OPTIONS_GET_PRIVATE(object);
+
+  switch (prop_id) {
+  case PROP_N_RETRIES:
+priv->n_retries = g_value_get_int(value);
+break;
+  default:
+G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec);
+break;
+  }
+}
+
+static void
+gplasma_client_options_get_property(GObject *object,
+guint prop_id,
+GValue *value,
+GParamSpec *pspec)
+{
+  auto priv = GPLASMA_CLIENT_OPTIONS_GET_PRIVATE(object);
+
+  switch (prop_id) {
+  case PROP_N_RETRIES:
+g_value_set_int(value, priv->n_retries);
+break;
+  default:
+G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec);
+break;
+  }
+}
+
+static void
+gplasma_client_options_init(GPlasmaClientOptions *object)
+{
+}
+
+static void
+gplasma_client_options_class_init(GPlasmaClientOptionsClass *klass)
+{
+  auto gobject_class = G_OBJECT_CLASS(klass);
+
+  gobject_class->set_property = gplasma_client_options_set_property;
+  gobject_class->get_property = gplasma_client_options_get_property;
+
+  GParamSpec *spec;
+  spec = g_param_spec_int("n-retries",
+  "N retries",
+  "The number of retries to connect plasma store. "
+  "-1 means that the system default value is used.",
+  -1,
+  G_MAXINT,
+  -1,
+  static_cast(G_PARAM_READWRITE |
+   G_PARAM_CONSTRUCT));
+  g_object_class_install_property(gobject_class, PROP_N_RETRIES, spec);
+}
+
+/**
+ * gplasma_client_options_new:
+ *
+ * Returns: A newly created #GPlasmaClientOptions.
+ *
+ * Since: 0.12.0
+ */
+GPlasmaClientOptions *
+gplasma_client_options_new(void)
+{
+  auto options = g_object_new(GPLASMA_TYPE_CLIENT_OPTIONS,
+  NULL);
+  return GPLASMA_CLIENT_OPTIONS(options);
+}
+
+/**
+ * gplasma_client_options_set_n_retries:
+ * @options: A #GPlasmaClientOptions.
+ * @n_retries: The number of retires on connect.
+ *
+ * Since: 0.12.0
+ */
+void
+gplasma_client_options_set_n_retries(GPlasmaClientOptions *options,
+ gint n_retries)
+{
+  auto priv = GPLASMA_CLIENT_OPTIONS_

[arrow] branch master updated: ARROW-4154: [GLib] Add GArrowDecimal128DataType

2019-01-05 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 857deae  ARROW-4154: [GLib] Add GArrowDecimal128DataType
857deae is described below

commit 857deae933478970b4fc0ff55fab61f32a5c6e4f
Author: Kouhei Sutou 
AuthorDate: Sat Jan 5 20:23:22 2019 +0900

ARROW-4154: [GLib] Add GArrowDecimal128DataType

garrow_decimal_data_type_new() is deprecated.

Author: Kouhei Sutou 

Closes #3305 from kou/glib-decimal128-data-type and squashes the following 
commits:

b51b7a19  Use decimal128
4823eea6   Add GArrowDecimal128DataType
---
 c_glib/arrow-glib/Makefile.am  |  6 +--
 c_glib/arrow-glib/array-builder.cpp|  6 +--
 c_glib/arrow-glib/array-builder.h  |  5 +-
 c_glib/arrow-glib/basic-array.cpp  |  4 +-
 c_glib/arrow-glib/basic-data-type.cpp  | 61 +-
 c_glib/arrow-glib/basic-data-type.h| 28 --
 c_glib/arrow-glib/{decimal.cpp => decimal128.cpp}  |  2 +-
 c_glib/arrow-glib/{decimal.h => decimal128.h}  |  0
 c_glib/arrow-glib/{decimal.hpp => decimal128.hpp}  |  2 +-
 c_glib/arrow-glib/meson.build  |  6 +--
 c_glib/arrow-glib/orc-file-reader.h|  2 +
 c_glib/doc/arrow-glib/arrow-glib-docs.xml  |  2 +-
 ...t-decimal-array.rb => test-decimal128-array.rb} |  6 +--
 ...l-data-type.rb => test-decimal128-data-type.rb} | 10 ++--
 .../test/{test-decimal.rb => test-decimal128.rb}   |  0
 15 files changed, 99 insertions(+), 41 deletions(-)

diff --git a/c_glib/arrow-glib/Makefile.am b/c_glib/arrow-glib/Makefile.am
index bf97168..a296595 100644
--- a/c_glib/arrow-glib/Makefile.am
+++ b/c_glib/arrow-glib/Makefile.am
@@ -59,7 +59,7 @@ libarrow_glib_la_headers =\
composite-array.h   \
composite-data-type.h   \
data-type.h \
-   decimal.h   \
+   decimal128.h\
error.h \
field.h \
gobject-type.h  \
@@ -110,7 +110,7 @@ libarrow_glib_la_sources =  \
column.cpp  \
composite-array.cpp \
composite-data-type.cpp \
-   decimal.cpp \
+   decimal128.cpp  \
error.cpp   \
field.cpp   \
record-batch.cpp\
@@ -155,7 +155,7 @@ libarrow_glib_la_cpp_headers =  \
codec.hpp   \
column.hpp  \
data-type.hpp   \
-   decimal.hpp \
+   decimal128.hpp  \
error.hpp   \
field.hpp   \
record-batch.hpp\
diff --git a/c_glib/arrow-glib/array-builder.cpp 
b/c_glib/arrow-glib/array-builder.cpp
index 4b61bfa..5f2d411 100644
--- a/c_glib/arrow-glib/array-builder.cpp
+++ b/c_glib/arrow-glib/array-builder.cpp
@@ -23,9 +23,9 @@
 
 #include 
 #include 
+#include 
 #include 
 #include 
-#include 
 
 template 
 gboolean
@@ -3803,14 +3803,14 @@ 
garrow_decimal128_array_builder_class_init(GArrowDecimal128ArrayBuilderClass *kl
 
 /**
  * garrow_decimal128_array_builder_new:
- * @data_type: #GArrowDecimalDataType for the decimal.
+ * @data_type: #GArrowDecimal128DataType for the decimal.
  *
  * Returns: A newly created #GArrowDecimal128ArrayBuilder.
  *
  * Since: 0.10.0
  */
 GArrowDecimal128ArrayBuilder *
-garrow_decimal128_array_builder_new(GArrowDecimalDataType *data_type)
+garrow_decimal128_array_builder_new(GArrowDecimal128DataType *data_type)
 {
   auto arrow_data_type = garrow_data_type_get_raw(GARROW_DATA_TYPE(data_type));
   auto builder = garrow_array_builder_new(arrow_data_type,
diff --git a/c_glib/arrow-glib/array-builder.h 
b/c_glib/arrow-glib/array-builder.h
index 1ddc026..b2ad6f4 100644
--- a/c_glib/arrow-glib/array-builder.h
+++ b/c_glib/arrow-glib/array-builder.h
@@ -20,8 +20,7 @@
 #pragma once
 
 #include 
-#include 
-#include 
+#include 
 
 G_BEGIN_DECLS
 
@@ -1475,7 +1474,7 @@ struct _GArrowDecimal128ArrayBuilderClass
   GArrowArrayBuilderClass parent_class;
 };
 
-GArrowDecimal128ArrayBuilder 
*garrow_decimal128_array_builder_new(GArrowDecimalDataType *data_type);
+GArrowDecimal128ArrayBuilder 
*garrow_decimal128_array_builder_new(GArrowDecimal128DataType *data_type);
 
 #ifndef GARROW_DISABLE_DEPRECATED
 GARROW_DEPR

[arrow] branch master updated: ARROW-4153: [GLib] Add builder_append_value() for consistency

2019-01-05 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new a4f4808  ARROW-4153: [GLib] Add builder_append_value() for consistency
a4f4808 is described below

commit a4f4808e274e46ce71b08188071d3e2db230c82e
Author: Kouhei Sutou 
AuthorDate: Sat Jan 5 18:55:26 2019 +0900

ARROW-4153: [GLib] Add builder_append_value() for consistency

Because we use builder_append_values() for multiple values.

builder_append() is deprecated.

Author: Kouhei Sutou 

Closes #3304 from kou/glib-builder-append-value and squashes the following 
commits:

e93c0419   Add builder_append_value() for consistency
---
 c_glib/arrow-cuda-glib/cuda.cpp   |   2 +-
 c_glib/arrow-glib/array-builder.cpp   | 626 +++---
 c_glib/arrow-glib/array-builder.h | 160 +
 c_glib/arrow-glib/codec.cpp   |   2 +-
 c_glib/arrow-glib/orc-file-reader.cpp |   3 +-
 c_glib/example/build.c|   6 +-
 c_glib/test/helper/buildable.rb   |   8 +-
 c_glib/test/test-array.rb |  18 +-
 c_glib/test/test-binary-array.rb  |  10 +-
 c_glib/test/test-boolean-array.rb |  14 +-
 c_glib/test/test-date32-array.rb  |  14 +-
 c_glib/test/test-date64-array.rb  |  14 +-
 c_glib/test/test-decimal-array.rb |   4 +-
 c_glib/test/test-double-array.rb  |  14 +-
 c_glib/test/test-float-array.rb   |  14 +-
 c_glib/test/test-int16-array.rb   |  14 +-
 c_glib/test/test-int32-array.rb   |  14 +-
 c_glib/test/test-int64-array.rb   |  14 +-
 c_glib/test/test-int8-array.rb|  14 +-
 c_glib/test/test-list-array.rb|  14 +-
 c_glib/test/test-string-array.rb  |   6 +-
 c_glib/test/test-struct-array.rb  |  12 +-
 c_glib/test/test-uint16-array.rb  |  14 +-
 c_glib/test/test-uint32-array.rb  |  14 +-
 c_glib/test/test-uint64-array.rb  |  14 +-
 c_glib/test/test-uint8-array.rb   |  14 +-
 26 files changed, 873 insertions(+), 180 deletions(-)

diff --git a/c_glib/arrow-cuda-glib/cuda.cpp b/c_glib/arrow-cuda-glib/cuda.cpp
index 3f82f8f..9679cc0 100644
--- a/c_glib/arrow-cuda-glib/cuda.cpp
+++ b/c_glib/arrow-cuda-glib/cuda.cpp
@@ -648,7 +648,7 @@ garrow_cuda_ipc_memory_handle_new(const guint8 *data,
  *
  * Returns: (transfer full): A newly created #GArrowBuffer on success,
  *   %NULL on error. The buffer has serialized @handle. The serialized
- *   @handle can be deserialized by garrow_gpu_cuda_ipc_memory_handle_new()
+ *   @handle can be deserialized by garrow_cuda_ipc_memory_handle_new()
  *   in other process.
  *
  * Since: 0.8.0
diff --git a/c_glib/arrow-glib/array-builder.cpp 
b/c_glib/arrow-glib/array-builder.cpp
index a5c7579..4b61bfa 100644
--- a/c_glib/arrow-glib/array-builder.cpp
+++ b/c_glib/arrow-glib/array-builder.cpp
@@ -29,10 +29,10 @@
 
 template 
 gboolean
-garrow_array_builder_append(GArrowArrayBuilder *builder,
-VALUE value,
-GError **error,
-const gchar *context)
+garrow_array_builder_append_value(GArrowArrayBuilder *builder,
+  VALUE value,
+  GError **error,
+  const gchar *context)
 {
   auto arrow_builder =
 static_cast(garrow_array_builder_get_raw(builder));
@@ -446,17 +446,38 @@ garrow_boolean_array_builder_new(void)
  * @error: (nullable): Return location for a #GError or %NULL.
  *
  * Returns: %TRUE on success, %FALSE if there was an error.
+ *
+ * Deprecated: 0.12.0:
+ *   Use garrow_boolean_array_builder_append_value() instead.
  */
 gboolean
 garrow_boolean_array_builder_append(GArrowBooleanArrayBuilder *builder,
 gboolean value,
 GError **error)
 {
-  return garrow_array_builder_append
+  return garrow_boolean_array_builder_append_value(builder, value, error);
+}
+
+/**
+ * garrow_boolean_array_builder_append_value:
+ * @builder: A #GArrowBooleanArrayBuilder.
+ * @value: A boolean value.
+ * @error: (nullable): Return location for a #GError or %NULL.
+ *
+ * Returns: %TRUE on success, %FALSE if there was an error.
+ *
+ * Since: 0.12.0
+ */
+gboolean
+garrow_boolean_array_builder_append_value(GArrowBooleanArrayBuilder *builder,
+  gboolean value,
+  GError **error)
+{
+  return garrow_array_builder_append_value
 (GARROW_ARRAY_BUILDER(builder),
  static_cast(value),
  error,
- "[boolean-array-builder][append]");
+ "[boolean-array-builder][append-value]");
 }
 
 /**
@@ -583,17 +604,38 @@ garrow_int_array_builder_new(void)
  * Returns: %TRUE on success, %FALSE if there was an error.
  *
  * Since: 0.6.0
+ *
+

[arrow] branch master updated: ARROW-4141: [Ruby] Add support for creating schema from raw Ruby objects

2019-01-03 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new c71d27f  ARROW-4141: [Ruby] Add support for creating schema from raw 
Ruby objects
c71d27f is described below

commit c71d27fe55ca2a273f194c860b59074b0c998a74
Author: Kouhei Sutou 
AuthorDate: Thu Jan 3 18:47:25 2019 +0900

ARROW-4141: [Ruby] Add support for creating schema from raw Ruby objects

The followings should be implemented by follow-up works:

  * Arrow::TimestampDataType.new(unit: ...)
  * Arrow::Time32DataType.new(unit: ...)
  * Arrow::Time64DataType.new(unit: ...)
  * Arrow::DecimalDataType.new(precision: ..., scale: ...)
  * Arrow::SparseUnionDataType.new(fields: ..., type_codes: ...)
  * Arrow::DenseUnionDataType.new(fields: ..., type_codes: ...)
  * Arrow::DictionaryDataType.new(fields: ..., type_codes: ...)

Author: Kouhei Sutou 

Closes #3293 from kou/ruby-schema-new and squashes the following commits:

d251ba9d  Add .yardopts to rat exclude files
169b8656   Add support for creating schema from raw Ruby 
objects
---
 dev/release/rat_exclude_files.txt  |   1 +
 ruby/red-arrow/.gitignore  |   2 +
 ruby/red-arrow/.yardopts   |   6 ++
 ruby/red-arrow/README.md   |   2 +-
 ruby/red-arrow/Rakefile|   4 +
 ruby/red-arrow/lib/arrow/data-type.rb  | 110 +++--
 ruby/red-arrow/lib/arrow/field.rb  |  99 +--
 ruby/red-arrow/lib/arrow/list-data-type.rb |  68 +
 ruby/red-arrow/lib/arrow/loader.rb |   1 +
 ruby/red-arrow/lib/arrow/schema.rb |  71 +
 ruby/red-arrow/lib/arrow/struct-data-type.rb   | 104 +++
 ruby/red-arrow/red-arrow.gemspec   |   2 +
 ruby/red-arrow/test/test-data-type.rb  |  47 +
 ruby/red-arrow/test/test-field.rb  |  71 +
 .../data-type.rb => test/test-list-data-type.rb}   |  42 
 ruby/red-arrow/test/test-schema.rb |  88 +
 ruby/red-arrow/test/test-struct-data-type.rb   |  96 ++
 17 files changed, 745 insertions(+), 69 deletions(-)

diff --git a/dev/release/rat_exclude_files.txt 
b/dev/release/rat_exclude_files.txt
index 7674e2f..1086793 100644
--- a/dev/release/rat_exclude_files.txt
+++ b/dev/release/rat_exclude_files.txt
@@ -186,5 +186,6 @@ r/README.md
 r/README.Rmd
 r/man/*.Rd
 .gitattributes
+ruby/red-arrow/.yardopts
 rust/test/data/*.csv
 rust/rust-toolchain
diff --git a/ruby/red-arrow/.gitignore b/ruby/red-arrow/.gitignore
index 779545d..68e4b5c 100644
--- a/ruby/red-arrow/.gitignore
+++ b/ruby/red-arrow/.gitignore
@@ -15,4 +15,6 @@
 # specific language governing permissions and limitations
 # under the License.
 
+/.yardoc/
+/doc/reference/
 /pkg/
diff --git a/ruby/red-arrow/.yardopts b/ruby/red-arrow/.yardopts
new file mode 100644
index 000..67159b1
--- /dev/null
+++ b/ruby/red-arrow/.yardopts
@@ -0,0 +1,6 @@
+--output-dir doc/reference
+--markup markdown
+--no-private
+lib/**/*.rb
+-
+doc/text/*
diff --git a/ruby/red-arrow/README.md b/ruby/red-arrow/README.md
index a6798dd..95ec396 100644
--- a/ruby/red-arrow/README.md
+++ b/ruby/red-arrow/README.md
@@ -39,7 +39,7 @@ Note that the Apache Arrow GLib packages are "unofficial". 
"Official" packages w
 
 Install Red Arrow after you install Apache Arrow GLib:
 
-```text
+```console
 % gem install red-arrow
 ```
 
diff --git a/ruby/red-arrow/Rakefile b/ruby/red-arrow/Rakefile
index 96851af..a3ece36 100644
--- a/ruby/red-arrow/Rakefile
+++ b/ruby/red-arrow/Rakefile
@@ -19,6 +19,7 @@
 
 require "rubygems"
 require "bundler/gem_helper"
+require "yard"
 
 base_dir = File.join(__dir__)
 
@@ -37,3 +38,6 @@ task :test do
 end
 
 task default: :test
+
+YARD::Rake::YardocTask.new do |task|
+end
diff --git a/ruby/red-arrow/lib/arrow/data-type.rb 
b/ruby/red-arrow/lib/arrow/data-type.rb
index dad74fb..03960e4 100644
--- a/ruby/red-arrow/lib/arrow/data-type.rb
+++ b/ruby/red-arrow/lib/arrow/data-type.rb
@@ -18,21 +18,117 @@
 module Arrow
   class DataType
 class << self
+  # Creates a new suitable {Arrow::DataType}.
+  #
+  # @overload resolve(data_type)
+  #
+  #   Returns the given data type itself. This is convenient to
+  #   use this method as {Arrow::DataType} converter.
+  #
+  #   @param data_type [Arrow::DataType] The data type.
+  #
+  #   @return [Arrow::DataType] The given data type itself.
+  #
+  # @overload resolve(name, *arguments)
+  #
+  #   Creates a suitable data type from type name. For example,
+

[arrow] branch master updated: ARROW-4132: [GLib] Add more GArrowTable constructors

2018-12-29 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 7074889  ARROW-4132: [GLib] Add more GArrowTable constructors
7074889 is described below

commit 7074889602a2279cfa2440697040a946628f5b56
Author: Kouhei Sutou 
AuthorDate: Sun Dec 30 09:56:05 2018 +0900

ARROW-4132: [GLib] Add more GArrowTable constructors

Author: Kouhei Sutou 

Closes #3285 from kou/glib-table-new and squashes the following commits:

8bab8046   Add more GArrowTable constructors
---
 c_glib/arrow-glib/composite-array.h |   2 +
 c_glib/arrow-glib/orc-file-reader.h |   4 +-
 c_glib/arrow-glib/table.cpp | 204 +++-
 c_glib/arrow-glib/table.h   |  33 +-
 c_glib/arrow-glib/version.h.in  |  23 
 c_glib/test/test-table.rb   |  61 +--
 6 files changed, 310 insertions(+), 17 deletions(-)

diff --git a/c_glib/arrow-glib/composite-array.h 
b/c_glib/arrow-glib/composite-array.h
index c634dbf..10432e2 100644
--- a/c_glib/arrow-glib/composite-array.h
+++ b/c_glib/arrow-glib/composite-array.h
@@ -130,8 +130,10 @@ GArrowStructArray *garrow_struct_array_new(GArrowDataType 
*data_type,
 GArrowArray *garrow_struct_array_get_field(GArrowStructArray *array,
gint i);
 
+#ifndef GARROW_DISABLE_DEPRECATED
 GARROW_DEPRECATED_IN_0_10_FOR(garrow_struct_array_flatten)
 GList *garrow_struct_array_get_fields(GArrowStructArray *array);
+#endif
 
 GARROW_AVAILABLE_IN_0_10
 GList *garrow_struct_array_flatten(GArrowStructArray *array, GError **error);
diff --git a/c_glib/arrow-glib/orc-file-reader.h 
b/c_glib/arrow-glib/orc-file-reader.h
index 9b2dbad..97cf1ef 100644
--- a/c_glib/arrow-glib/orc-file-reader.h
+++ b/c_glib/arrow-glib/orc-file-reader.h
@@ -39,7 +39,7 @@ garrow_orc_file_reader_new(GArrowSeekableInputStream *file,
GError **error);
 
 #ifndef GARROW_DISABLE_DEPRECATED
-G_GNUC_DEPRECATED_FOR(garrow_orc_file_reader_set_field_indices)
+GARROW_DEPRECATED_IN_0_12_FOR(garrow_orc_file_reader_set_field_indices)
 void
 garrow_orc_file_reader_set_field_indexes(GArrowORCFileReader *reader,
  const gint *field_indexes,
@@ -50,7 +50,7 @@ garrow_orc_file_reader_set_field_indices(GArrowORCFileReader 
*reader,
  const gint *field_indices,
  guint n_field_indices);
 #ifndef GARROW_DISABLE_DEPRECATED
-G_GNUC_DEPRECATED_FOR(garrow_orc_file_reader_get_field_indices)
+GARROW_DEPRECATED_IN_0_12_FOR(garrow_orc_file_reader_get_field_indices)
 const gint *
 garrow_orc_file_reader_get_field_indexes(GArrowORCFileReader *reader,
  guint *n_field_indexes);
diff --git a/c_glib/arrow-glib/table.cpp b/c_glib/arrow-glib/table.cpp
index f9e1b95..b889eb2 100644
--- a/c_glib/arrow-glib/table.cpp
+++ b/c_glib/arrow-glib/table.cpp
@@ -21,8 +21,10 @@
 #  include 
 #endif
 
+#include 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -133,23 +135,219 @@ garrow_table_class_init(GArrowTableClass *klass)
  * @columns: (element-type GArrowColumn): The columns of the table.
  *
  * Returns: A newly created #GArrowTable.
+ *
+ * Deprecated: 0.12.0: Use garrow_table_new_values() instead.
  */
 GArrowTable *
 garrow_table_new(GArrowSchema *schema,
  GList *columns)
 {
+  auto arrow_schema = garrow_schema_get_raw(schema);
   std::vector> arrow_columns;
   for (GList *node = columns; node; node = node->next) {
-GArrowColumn *column = GARROW_COLUMN(node->data);
+auto column = GARROW_COLUMN(node->data);
 arrow_columns.push_back(garrow_column_get_raw(column));
   }
 
-  auto arrow_table =
-arrow::Table::Make(garrow_schema_get_raw(schema), arrow_columns);
+  auto arrow_table = arrow::Table::Make(arrow_schema, arrow_columns);
   return garrow_table_new_raw(_table);
 }
 
 /**
+ * garrow_table_new_values: (skip)
+ * @schema: The schema of the table.
+ * @values: The values of the table. All values must be instance of the
+ *   same class. Available classes are #GArrowColumn, #GArrowArray and
+ *   #GArrowRecordBatch.
+ * @error: (nullable): Return location for a #GError or %NULL.
+ *
+ * Returns: (nullable): A newly created #GArrowTable or %NULL on error.
+ *
+ * Since: 0.12.0
+ */
+GArrowTable *
+garrow_table_new_values(GArrowSchema *schema,
+GList *values,
+GError **error)
+{
+  const auto context = "[table][new][values]";
+  auto arrow_schema = garrow_schema_get_raw(schema);
+  std::vector> arrow_columns;
+  std::vector> arrow_arrays;
+  std::vector> arrow_record_batches;
+  for (GList *node = values; node; node = node->next) {
+if (GARROW_IS_COLUMN(node->data))

[arrow] branch master updated: ARROW-4085: [GLib] Use "field" for struct data type

2018-12-20 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 729cc3d  ARROW-4085: [GLib] Use "field" for struct data type
729cc3d is described below

commit 729cc3d3f31ebeeab9a86ec0ed59cf4000802135
Author: Kouhei Sutou 
AuthorDate: Thu Dec 20 18:27:01 2018 +0900

ARROW-4085: [GLib] Use "field" for struct data type

Because C++ API is changed to use "field" by ARROW-3545.

Author: Kouhei Sutou 

Closes #3229 from kou/glib-use-field and squashes the following commits:

c078e31f   Use "field" for struct data type
---
 c_glib/arrow-glib/composite-data-type.cpp | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/c_glib/arrow-glib/composite-data-type.cpp 
b/c_glib/arrow-glib/composite-data-type.cpp
index a4d3d84..599506f 100644
--- a/c_glib/arrow-glib/composite-data-type.cpp
+++ b/c_glib/arrow-glib/composite-data-type.cpp
@@ -230,7 +230,7 @@ 
garrow_struct_data_type_get_field_by_name(GArrowStructDataType *data_type,
   auto arrow_struct_data_type =
 std::static_pointer_cast(arrow_data_type);
 
-  auto arrow_field = arrow_struct_data_type->GetChildByName(name);
+  auto arrow_field = arrow_struct_data_type->GetFieldByName(name);
   if (arrow_field) {
 return garrow_field_new_raw(_field);
   } else {
@@ -256,7 +256,7 @@ 
garrow_struct_data_type_get_field_index(GArrowStructDataType *data_type,
   auto arrow_struct_data_type =
 std::static_pointer_cast(arrow_data_type);
 
-  return arrow_struct_data_type->GetChildIndex(name);
+  return arrow_struct_data_type->GetFieldIndex(name);
 }

[arrow] branch master updated: ARROW-4034: [Ruby] Add support :append option to FileOutputStream

2018-12-16 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 5d1934f  ARROW-4034: [Ruby] Add support :append option to 
FileOutputStream
5d1934f is described below

commit 5d1934fc3f5c65f70a3966b71c68941b2fd8d362
Author: Kouhei Sutou 
AuthorDate: Mon Dec 17 09:57:20 2018 +0900

ARROW-4034: [Ruby] Add support :append option to FileOutputStream

Author: Kouhei Sutou 

Closes #3193 from kou/ruby-file-output-stream-append and squashes the 
following commits:

6240f4b7   Add support :append option to FileOutputStream
---
 ruby/red-arrow/lib/arrow/field.rb  |  1 +
 .../lib/arrow/{field.rb => file-output-stream.rb}  | 22 -
 ruby/red-arrow/lib/arrow/loader.rb |  1 +
 ruby/red-arrow/lib/arrow/table.rb  |  1 +
 ruby/red-arrow/test/test-file-output-stream.rb | 54 ++
 5 files changed, 68 insertions(+), 11 deletions(-)

diff --git a/ruby/red-arrow/lib/arrow/field.rb 
b/ruby/red-arrow/lib/arrow/field.rb
index b1ed114..be5865f 100644
--- a/ruby/red-arrow/lib/arrow/field.rb
+++ b/ruby/red-arrow/lib/arrow/field.rb
@@ -18,6 +18,7 @@
 module Arrow
   class Field
 alias_method :initialize_raw, :initialize
+private :initialize_raw
 def initialize(name, data_type)
   case data_type
   when String, Symbol
diff --git a/ruby/red-arrow/lib/arrow/field.rb 
b/ruby/red-arrow/lib/arrow/file-output-stream.rb
similarity index 66%
copy from ruby/red-arrow/lib/arrow/field.rb
copy to ruby/red-arrow/lib/arrow/file-output-stream.rb
index b1ed114..f39ad14 100644
--- a/ruby/red-arrow/lib/arrow/field.rb
+++ b/ruby/red-arrow/lib/arrow/file-output-stream.rb
@@ -16,19 +16,19 @@
 # under the License.
 
 module Arrow
-  class Field
+  class FileOutputStream
 alias_method :initialize_raw, :initialize
-def initialize(name, data_type)
-  case data_type
-  when String, Symbol
-data_type_name = data_type.to_s.capitalize.gsub(/\AUint/, "UInt")
-data_type_class_name = "#{data_type_name}DataType"
-if Arrow.const_defined?(data_type_class_name)
-  data_type_class = Arrow.const_get(data_type_class_name)
-  data_type = data_type_class.new
-end
+private :initialize_raw
+def initialize(path, options={})
+  append = nil
+  case options
+  when true, false
+append = options
+  when Hash
+append = options[:append]
   end
-  initialize_raw(name, data_type)
+  append = false if append.nil?
+  initialize_raw(path, append)
 end
   end
 end
diff --git a/ruby/red-arrow/lib/arrow/loader.rb 
b/ruby/red-arrow/lib/arrow/loader.rb
index 736f25b..2092e46 100644
--- a/ruby/red-arrow/lib/arrow/loader.rb
+++ b/ruby/red-arrow/lib/arrow/loader.rb
@@ -44,6 +44,7 @@ module Arrow
   require "arrow/date64-array"
   require "arrow/date64-array-builder"
   require "arrow/field"
+  require "arrow/file-output-stream"
   require "arrow/path-extension"
   require "arrow/record"
   require "arrow/record-batch"
diff --git a/ruby/red-arrow/lib/arrow/table.rb 
b/ruby/red-arrow/lib/arrow/table.rb
index 524517f..69a1de3 100644
--- a/ruby/red-arrow/lib/arrow/table.rb
+++ b/ruby/red-arrow/lib/arrow/table.rb
@@ -29,6 +29,7 @@ module Arrow
 end
 
 alias_method :initialize_raw, :initialize
+private :initialize_raw
 def initialize(schema_or_raw_table_or_columns, columns=nil)
   if columns.nil?
 if schema_or_raw_table_or_columns[0].is_a?(Column)
diff --git a/ruby/red-arrow/test/test-file-output-stream.rb 
b/ruby/red-arrow/test/test-file-output-stream.rb
new file mode 100644
index 000..559406a
--- /dev/null
+++ b/ruby/red-arrow/test/test-file-output-stream.rb
@@ -0,0 +1,54 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+class TestFileOutputStream < Test::Unit::TestCase
+  sub_test_case(".open") do
+def setup
+  @file = Tempfile

[arrow] branch master updated: ARROW-4004: [GLib] Replace GPU with CUDA

2018-12-12 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new c0ac97f  ARROW-4004: [GLib] Replace GPU with CUDA
c0ac97f is described below

commit c0ac97f126c98fb29e81d6544adfea9d4ab74aff
Author: Kouhei Sutou 
AuthorDate: Wed Dec 12 19:57:12 2018 +0900

ARROW-4004: [GLib] Replace GPU with CUDA

This is a follow-up change for #3088.

Author: Kouhei Sutou 

Closes #3162 from kou/glib-replace-gpu-with-cuda and squashes the following 
commits:

8891e510   Replace GPU with CUDA
---
 c_glib/plasma-glib/plasma-glib.pc.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/c_glib/plasma-glib/plasma-glib.pc.in 
b/c_glib/plasma-glib/plasma-glib.pc.in
index f3a82c2..c82fe69 100644
--- a/c_glib/plasma-glib/plasma-glib.pc.in
+++ b/c_glib/plasma-glib/plasma-glib.pc.in
@@ -25,4 +25,4 @@ Description: C API for Apache Arrow Plasma based on GLib
 Version: @VERSION@
 Libs: -L${libdir} -lplasma-glib
 Cflags: -I${includedir}
-Requires: plasma arrow-glib @ARROW_GPU_GLIB_PACKAGE@
+Requires: plasma arrow-glib @ARROW_CUDA_GLIB_PACKAGE@

[arrow] branch master updated: ARROW-3946: [GLib] Add support for union

2018-12-06 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 494ef75  ARROW-3946: [GLib] Add support for union
494ef75 is described below

commit 494ef75f4989064210a699b0e3c715a7d67d07fa
Author: Kouhei Sutou 
AuthorDate: Fri Dec 7 10:25:34 2018 +0900

ARROW-3946: [GLib] Add support for union

Author: Kouhei Sutou 

Closes #3112 from kou/glib-union-array and squashes the following commits:

b1d8870f  Fix a typo
c45e7fc0   Add support for union
---
 c_glib/arrow-glib/basic-array.cpp  |  15 +-
 c_glib/arrow-glib/basic-data-type.cpp  |  11 +
 c_glib/arrow-glib/composite-array.cpp  | 171 ++-
 c_glib/arrow-glib/composite-array.h|  52 -
 c_glib/arrow-glib/composite-data-type.cpp  | 238 -
 c_glib/arrow-glib/composite-data-type.h|  60 ++
 c_glib/test/test-dense-union-array.rb  |  50 +
 .../test/test-dense-union-data-type.rb |  56 ++---
 c_glib/test/test-sparse-union-array.rb |  49 +
 .../test/test-sparse-union-data-type.rb|  56 ++---
 ruby/red-arrow/lib/arrow/array.rb  |   6 +-
 11 files changed, 661 insertions(+), 103 deletions(-)

diff --git a/c_glib/arrow-glib/basic-array.cpp 
b/c_glib/arrow-glib/basic-array.cpp
index 77f64fc..47f9a95 100644
--- a/c_glib/arrow-glib/basic-array.cpp
+++ b/c_glib/arrow-glib/basic-array.cpp
@@ -209,7 +209,9 @@ enum {
   PROP_ARRAY
 };
 
-G_DEFINE_TYPE_WITH_PRIVATE(GArrowArray, garrow_array, G_TYPE_OBJECT)
+G_DEFINE_ABSTRACT_TYPE_WITH_PRIVATE(GArrowArray,
+garrow_array,
+G_TYPE_OBJECT)
 
 #define GARROW_ARRAY_GET_PRIVATE(obj) \
   static_cast(  \
@@ -2255,6 +2257,17 @@ garrow_array_new_raw(std::shared_ptr 
*arrow_array)
   case arrow::Type::type::STRUCT:
 type = GARROW_TYPE_STRUCT_ARRAY;
 break;
+  case arrow::Type::type::UNION:
+{
+  auto arrow_union_array =
+std::static_pointer_cast(*arrow_array);
+  if (arrow_union_array->mode() == arrow::UnionMode::SPARSE) {
+type = GARROW_TYPE_SPARSE_UNION_ARRAY;
+  } else {
+type = GARROW_TYPE_DENSE_UNION_ARRAY;
+  }
+}
+break;
   case arrow::Type::type::DICTIONARY:
 type = GARROW_TYPE_DICTIONARY_ARRAY;
 break;
diff --git a/c_glib/arrow-glib/basic-data-type.cpp 
b/c_glib/arrow-glib/basic-data-type.cpp
index 24133c9..86b86cf 100644
--- a/c_glib/arrow-glib/basic-data-type.cpp
+++ b/c_glib/arrow-glib/basic-data-type.cpp
@@ -1184,6 +1184,17 @@ 
garrow_data_type_new_raw(std::shared_ptr *arrow_data_type)
   case arrow::Type::type::STRUCT:
 type = GARROW_TYPE_STRUCT_DATA_TYPE;
 break;
+  case arrow::Type::type::UNION:
+{
+  auto arrow_union_data_type =
+std::static_pointer_cast(*arrow_data_type);
+  if (arrow_union_data_type->mode() == arrow::UnionMode::SPARSE) {
+type = GARROW_TYPE_SPARSE_UNION_DATA_TYPE;
+  } else {
+type = GARROW_TYPE_DENSE_UNION_DATA_TYPE;
+  }
+}
+break;
   case arrow::Type::type::DICTIONARY:
 type = GARROW_TYPE_DICTIONARY_DATA_TYPE;
 break;
diff --git a/c_glib/arrow-glib/composite-array.cpp 
b/c_glib/arrow-glib/composite-array.cpp
index b040ac7..bff1858 100644
--- a/c_glib/arrow-glib/composite-array.cpp
+++ b/c_glib/arrow-glib/composite-array.cpp
@@ -41,10 +41,18 @@ G_BEGIN_DECLS
  * use #GArrowListArrayBuilder to create a new array.
  *
  * #GArrowStructArray is a class for struct array. It can store zero
- * or more structs. One struct has zero or more fields. If you don't
+ * or more structs. One struct has one or more fields. If you don't
  * have Arrow format data, you need to use #GArrowStructArrayBuilder
  * to create a new array.
  *
+ * #GArrowUnionArray is a base class for union array. It can store
+ * zero or more unions. One union has one or more fields but one union
+ * can store only one field value.
+ *
+ * #GArrowDenseUnionArray is a class for dense union array.
+ *
+ * #GArrowSparseUnionArray is a class for sparse union array.
+ *
  * #GArrowDictionaryArray is a class for dictionary array. It can
  * store data with dictionary and indices. It's space effective than
  * normal array when the array has many same values. You can convert a
@@ -159,7 +167,7 @@ garrow_struct_array_class_init(GArrowStructArrayClass 
*klass)
  * garrow_struct_array_new:
  * @data_type: The data type of the struct.
  * @length: The number of elements.
- * @children: (element-type GArrowArray): The arrays for each field
+ * @fields: (element-type GArrowArray): The arrays for each field
  *   as #GList of #GArrowArray.
  * @null_bitmap: (nullable): The bitmap that shows null elements. The
  *   N-th element i

[arrow] branch master updated: ARROW-3912: [Plasma][GLib] Add support for creating and referring objects

2018-12-02 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 8f02a1b  ARROW-3912: [Plasma][GLib] Add support for creating and 
referring objects
8f02a1b is described below

commit 8f02a1b28ac10a2baa7250e00806d8663daaa3d6
Author: Kouhei Sutou 
AuthorDate: Sun Dec 2 19:39:18 2018 +0900

ARROW-3912: [Plasma][GLib] Add support for creating and referring objects

Author: Kouhei Sutou 

Closes #3056 from kou/glib-plasma-create-get and squashes the following 
commits:

e6bc59ff  Add missing status check for Disconnect()
623ce411  Add missing return on error
d90d8960  Add missing status check for Release()
1d5bf107  Add missing include
7ac27db6  Support old GObject Introspection
84df1849  Support old GObject Introspection
26b2926e   Add support for creating and referring objects
---
 c_glib/arrow-gpu-glib/meson.build  |  36 +-
 c_glib/configure.ac|   4 +
 c_glib/doc/plasma-glib/Makefile.am |   7 +
 c_glib/doc/plasma-glib/meson.build |   3 +
 c_glib/doc/plasma-glib/plasma-glib-docs.xml|   8 +-
 c_glib/plasma-glib/Makefile.am |  59 ++-
 c_glib/plasma-glib/client.cpp  | 330 -
 c_glib/plasma-glib/client.h|  36 +-
 c_glib/plasma-glib/client.hpp  |   8 +-
 c_glib/plasma-glib/meson.build |  54 ++-
 c_glib/plasma-glib/object.cpp  | 538 +
 c_glib/plasma-glib/object.h|  89 
 c_glib/plasma-glib/{client.hpp => object.hpp}  |  22 +-
 c_glib/plasma-glib/plasma-glib.h   |   1 +
 c_glib/plasma-glib/plasma-glib.hpp |   1 +
 c_glib/plasma-glib/plasma-glib.pc.in   |   2 +-
 c_glib/test/plasma/test-plasma-client.rb   |  58 ++-
 ...sma-client.rb => test-plasma-created-object.rb} |  29 +-
 ...ma-client.rb => test-plasma-referred-object.rb} |  24 +-
 19 files changed, 1230 insertions(+), 79 deletions(-)

diff --git a/c_glib/arrow-gpu-glib/meson.build 
b/c_glib/arrow-gpu-glib/meson.build
index e6b170e..680982e 100644
--- a/c_glib/arrow-gpu-glib/meson.build
+++ b/c_glib/arrow-gpu-glib/meson.build
@@ -57,19 +57,23 @@ pkgconfig.generate(filebase: 'arrow-gpu-glib',
requires: ['arrow-glib', 'arrow-gpu'],
libraries: [libarrow_gpu_glib])
 
-gnome.generate_gir(libarrow_gpu_glib,
-   dependencies: declare_dependency(sources: arrow_glib_gir),
-   sources: sources + c_headers,
-   namespace: 'ArrowGPU',
-   nsversion: api_version,
-   identifier_prefix: 'GArrowGPU',
-   symbol_prefix: 'garrow_gpu',
-   export_packages: 'arrow-gpu-glib',
-   includes: [
- 'Arrow-1.0',
-   ],
-   install: true,
-   extra_args: [
- '--warn-all',
- '--include-uninstalled=./arrow-glib/Arrow-1.0.gir',
-   ])
+gir_dependencies = [
+  declare_dependency(sources: arrow_glib_gir),
+]
+gir_extra_args = [
+  '--warn-all',
+  '--include-uninstalled=./arrow-glib/Arrow-1.0.gir',
+]
+arrow_gpu_glib_gir = gnome.generate_gir(libarrow_gpu_glib,
+dependencies: gir_dependencies,
+sources: sources + c_headers,
+namespace: 'ArrowGPU',
+nsversion: api_version,
+identifier_prefix: 'GArrowGPU',
+symbol_prefix: 'garrow_gpu',
+export_packages: 'arrow-gpu-glib',
+includes: [
+  'Arrow-1.0',
+],
+install: true,
+extra_args: gir_extra_args)
diff --git a/c_glib/configure.ac b/c_glib/configure.ac
index badf9e9..b84e3d3 100644
--- a/c_glib/configure.ac
+++ b/c_glib/configure.ac
@@ -223,8 +223,12 @@ fi
 
 AM_CONDITIONAL([HAVE_ARROW_GPU], [test "$HAVE_ARROW_GPU" = "yes"])
 if test "$HAVE_ARROW_GPU" = "yes"; then
+  ARROW_GPU_GLIB_PACKAGE="arrow-gpu-glib"
   AC_DEFINE(HAVE_ARROW_GPU, [1], [Define to 1 if Apache Arrow supports GPU.])
+else
+  ARROW_GPU_GLIB_PACKAGE=""
 fi
+AC_SUBST(ARROW_GPU_GLIB_PACKAGE)
 
 AM_CONDITIONAL([HAVE_GANDIVA], [test "$HAVE_GANDIVA" = "yes"])
 if test "$HAVE_GA

[arrow] branch master updated: ARROW-3900: [GLib] Add garrow_mutable_buffer_set_data()

2018-11-29 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 2d089a6  ARROW-3900: [GLib] Add garrow_mutable_buffer_set_data()
2d089a6 is described below

commit 2d089a67aa780e0b54d6e5ba42d04bb083ab5fe3
Author: Kouhei Sutou 
AuthorDate: Thu Nov 29 23:04:31 2018 +0900

ARROW-3900: [GLib] Add garrow_mutable_buffer_set_data()

Author: Kouhei Sutou 

Closes #3052 from kou/glib-mutable-buffer-set-data and squashes the 
following commits:

2d724912   Add garrow_mutable_buffer_set_data()
---
 c_glib/arrow-glib/buffer.cpp   | 38 ++
 c_glib/arrow-glib/buffer.h |  5 +
 c_glib/test/test-mutable-buffer.rb | 26 ++
 3 files changed, 69 insertions(+)

diff --git a/c_glib/arrow-glib/buffer.cpp b/c_glib/arrow-glib/buffer.cpp
index 9ba98f0..90d518b 100644
--- a/c_glib/arrow-glib/buffer.cpp
+++ b/c_glib/arrow-glib/buffer.cpp
@@ -485,6 +485,44 @@ garrow_mutable_buffer_slice(GArrowMutableBuffer *buffer,
   return garrow_mutable_buffer_new_raw_bytes(_buffer, priv->data);
 }
 
+/**
+ * garrow_mutable_buffer_set_data:
+ * @buffer: A #GArrowMutableBuffer.
+ * @offset: A write offset in the buffer data in byte.
+ * @data: (array length=size): The data to be written.
+ * @size: The number of bytes of the data to be written.
+ * @error: (nullable): Return location for a #GError or %NULL.
+ *
+ * Returns: %TRUE on success, %FALSE otherwise.
+ *
+ * Since: 0.12.0
+ */
+gboolean
+garrow_mutable_buffer_set_data(GArrowMutableBuffer *buffer,
+   gint64 offset,
+   const guint8 *data,
+   gint64 size,
+   GError **error)
+{
+  const gchar *context = "[mutable-buffer][set-data]";
+  auto arrow_buffer = garrow_buffer_get_raw(GARROW_BUFFER(buffer));
+  if (offset + size > arrow_buffer->size()) {
+g_set_error(error,
+GARROW_ERROR,
+GARROW_ERROR_INVALID,
+"%s: Data is too large: "
+"<(%" G_GINT64_FORMAT " + %" G_GINT64_FORMAT ") > "
+"(%" G_GINT64_FORMAT ")>",
+context,
+offset,
+size,
+arrow_buffer->size());
+return FALSE;
+  }
+  memcpy(arrow_buffer->mutable_data() + offset, data, size);
+  return TRUE;
+}
+
 
 G_DEFINE_TYPE(GArrowResizableBuffer,
   garrow_resizable_buffer,
diff --git a/c_glib/arrow-glib/buffer.h b/c_glib/arrow-glib/buffer.h
index c27699a..a176071 100644
--- a/c_glib/arrow-glib/buffer.h
+++ b/c_glib/arrow-glib/buffer.h
@@ -75,6 +75,11 @@ GArrowMutableBuffer *garrow_mutable_buffer_new_bytes(GBytes 
*data);
 GArrowMutableBuffer *garrow_mutable_buffer_slice(GArrowMutableBuffer *buffer,
  gint64 offset,
  gint64 size);
+gboolean garrow_mutable_buffer_set_data(GArrowMutableBuffer *buffer,
+gint64 offset,
+const guint8 *data,
+gint64 size,
+GError **error);
 
 
 #define GARROW_TYPE_RESIZABLE_BUFFER (garrow_resizable_buffer_get_type())
diff --git a/c_glib/test/test-mutable-buffer.rb 
b/c_glib/test/test-mutable-buffer.rb
index f370e60..ccfd6e2 100644
--- a/c_glib/test/test-mutable-buffer.rb
+++ b/c_glib/test/test-mutable-buffer.rb
@@ -45,4 +45,30 @@ class TestMutableBuffer < Test::Unit::TestCase
 sliced_buffer = @buffer.slice(1, 3)
 assert_equal(@data[1, 3], sliced_buffer.data.to_s)
   end
+
+  sub_test_case("#set_data") do
+test("offset") do
+  @buffer.set_data(1, "EL")
+  assert_equal("HELlo", @buffer.data.to_s)
+end
+
+test("replace") do
+  @buffer.set_data(0, "World")
+  assert_equal("World", @buffer.data.to_s)
+end
+
+test("offset: too large") do
+  message = "[mutable-buffer][set-data]: Data is too large: <(5 + 1) > 
(5)>"
+  assert_raise(Arrow::Error::Invalid.new(message)) do
+@buffer.set_data(5, "X")
+  end
+end
+
+test("data too large") do
+  message = "[mutable-buffer][set-data]: Data is too large: <(0 + 6) > 
(5)>"
+  assert_raise(Arrow::Error::Invalid.new(message)) do
+@buffer.set_data(0, @data + "!")
+  end
+end
+  end
 end

[arrow] branch master updated: ARROW-3864: [GLib] Add support for allow-float-truncate cast option

2018-11-24 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 6dbd1ec  ARROW-3864: [GLib] Add support for allow-float-truncate cast 
option
6dbd1ec is described below

commit 6dbd1ecfd081a7366068f1bcdbece612706ad6ed
Author: Kouhei Sutou 
AuthorDate: Sat Nov 24 22:49:29 2018 +0900

ARROW-3864: [GLib] Add support for allow-float-truncate cast option

Author: Kouhei Sutou 

Closes #3023 from kou/glib-add-allow-float-truncate-cast-options and 
squashes the following commits:

57611105   Add support for allow-float-truncate cast option
---
 c_glib/arrow-glib/compute.cpp | 28 +---
 c_glib/test/test-cast.rb  | 17 +
 2 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/c_glib/arrow-glib/compute.cpp b/c_glib/arrow-glib/compute.cpp
index ce427e6..2039eea 100644
--- a/c_glib/arrow-glib/compute.cpp
+++ b/c_glib/arrow-glib/compute.cpp
@@ -41,7 +41,8 @@ typedef struct GArrowCastOptionsPrivate_ {
 enum {
   PROP_0,
   PROP_ALLOW_INT_OVERFLOW,
-  PROP_ALLOW_TIME_TRUNCATE
+  PROP_ALLOW_TIME_TRUNCATE,
+  PROP_ALLOW_FLOAT_TRUNCATE
 };
 
 G_DEFINE_TYPE_WITH_PRIVATE(GArrowCastOptions,
@@ -68,6 +69,9 @@ garrow_cast_options_set_property(GObject *object,
   case PROP_ALLOW_TIME_TRUNCATE:
 priv->options.allow_time_truncate = g_value_get_boolean(value);
 break;
+  case PROP_ALLOW_FLOAT_TRUNCATE:
+priv->options.allow_float_truncate = g_value_get_boolean(value);
+break;
   default:
 G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec);
 break;
@@ -89,6 +93,9 @@ garrow_cast_options_get_property(GObject *object,
   case PROP_ALLOW_TIME_TRUNCATE:
 g_value_set_boolean(value, priv->options.allow_time_truncate);
 break;
+  case PROP_ALLOW_FLOAT_TRUNCATE:
+g_value_set_boolean(value, priv->options.allow_float_truncate);
+break;
   default:
 G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec);
 break;
@@ -103,13 +110,12 @@ garrow_cast_options_init(GArrowCastOptions *object)
 static void
 garrow_cast_options_class_init(GArrowCastOptionsClass *klass)
 {
-  GParamSpec *spec;
-
   auto gobject_class = G_OBJECT_CLASS(klass);
 
   gobject_class->set_property = garrow_cast_options_set_property;
   gobject_class->get_property = garrow_cast_options_get_property;
 
+  GParamSpec *spec;
   /**
* GArrowCastOptions:allow-int-overflow:
*
@@ -137,6 +143,20 @@ garrow_cast_options_class_init(GArrowCastOptionsClass 
*klass)
   FALSE,
   static_cast(G_PARAM_READWRITE));
   g_object_class_install_property(gobject_class, PROP_ALLOW_TIME_TRUNCATE, 
spec);
+
+  /**
+   * GArrowCastOptions:allow-float-truncate:
+   *
+   * Whether truncating float value is allowed or not.
+   *
+   * Since: 0.12.0
+   */
+  spec = g_param_spec_boolean("allow-float-truncate",
+  "Allow float truncate",
+  "Whether truncating float value is allowed or 
not",
+  FALSE,
+  static_cast(G_PARAM_READWRITE));
+  g_object_class_install_property(gobject_class, PROP_ALLOW_FLOAT_TRUNCATE, 
spec);
 }
 
 /**
@@ -161,6 +181,8 @@ garrow_cast_options_new_raw(arrow::compute::CastOptions 
*arrow_cast_options)
   auto cast_options =
 g_object_new(GARROW_TYPE_CAST_OPTIONS,
  "allow-int-overflow", arrow_cast_options->allow_int_overflow,
+ "allow-time-truncate", 
arrow_cast_options->allow_time_truncate,
+ "allow-float-truncate", 
arrow_cast_options->allow_float_truncate,
  NULL);
   return GARROW_CAST_OPTIONS(cast_options);
 }
diff --git a/c_glib/test/test-cast.rb b/c_glib/test/test-cast.rb
index 6c29e85..2512e05 100644
--- a/c_glib/test/test-cast.rb
+++ b/c_glib/test/test-cast.rb
@@ -65,4 +65,21 @@ class TestCast < Test::Unit::TestCase
milli_array.cast(second_timestamp, options))
 end
   end
+
+  sub_test_case("allow-float-truncate") do
+def test_default
+  require_gi(1, 42, 0)
+  assert_raise(Arrow::Error::Invalid) do
+build_float_array([1.1]).cast(Arrow::Int8DataType.new)
+  end
+end
+
+def test_true
+  options = Arrow::CastOptions.new
+  options.allow_float_truncate = true
+  int8_data_type = Arrow::Int8DataType.new
+  assert_equal(build_int8_array([1]),
+   build_float_array([1.1]).cast(int8_data_type, options))
+end
+  end
 end

[arrow] branch master updated: ARROW-3798: [GLib] Add support for column type CSV read option

2018-11-15 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 281eb22  ARROW-3798: [GLib] Add support for column type CSV read option
281eb22 is described below

commit 281eb22f8cb7f17afcdabf3795177c25063f4888
Author: Kouhei Sutou 
AuthorDate: Thu Nov 15 21:21:15 2018 +0900

ARROW-3798: [GLib] Add support for column type CSV read option

Author: Kouhei Sutou 

Closes #2973 from kou/glib-csv-type and squashes the following commits:

3cb0d078   Add column type CSV read option
---
 c_glib/arrow-glib/reader.cpp   | 68 ++
 c_glib/arrow-glib/reader.h |  9 ++
 c_glib/test/test-csv-reader.rb | 64 ++-
 3 files changed, 127 insertions(+), 14 deletions(-)

diff --git a/c_glib/arrow-glib/reader.cpp b/c_glib/arrow-glib/reader.cpp
index 5253a45..b4b5c08 100644
--- a/c_glib/arrow-glib/reader.cpp
+++ b/c_glib/arrow-glib/reader.cpp
@@ -22,6 +22,7 @@
 #endif
 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -1276,6 +1277,73 @@ garrow_csv_read_options_new(void)
   return GARROW_CSV_READ_OPTIONS(csv_read_options);
 }
 
+/**
+ * garrow_csv_read_options_add_column_type:
+ * @options: A #GArrowCSVReadOptions.
+ * @name: The name of the target column.
+ * @data_type: The #GArrowDataType for the column.
+ *
+ * Add value type of a column.
+ *
+ * Since: 0.12.0
+ */
+void
+garrow_csv_read_options_add_column_type(GArrowCSVReadOptions *options,
+const gchar *name,
+GArrowDataType *data_type)
+{
+  auto priv = GARROW_CSV_READ_OPTIONS_GET_PRIVATE(options);
+  auto arrow_data_type = garrow_data_type_get_raw(data_type);
+  priv->convert_options.column_types[name] = arrow_data_type;
+}
+
+/**
+ * garrow_csv_read_options_add_schema:
+ * @options: A #GArrowCSVReadOptions.
+ * @schema: The #GArrowSchema that specifies columns and their types.
+ *
+ * Add value types for columns in the schema.
+ *
+ * Since: 0.12.0
+ */
+void
+garrow_csv_read_options_add_schema(GArrowCSVReadOptions *options,
+   GArrowSchema *schema)
+{
+  auto priv = GARROW_CSV_READ_OPTIONS_GET_PRIVATE(options);
+  auto arrow_schema = garrow_schema_get_raw(schema);
+  for (const auto field : arrow_schema->fields()) {
+priv->convert_options.column_types[field->name()] = field->type();
+  }
+}
+
+/**
+ * garrow_csv_read_options_get_column_types:
+ * @options: A #GArrowCSVReadOptions.
+ *
+ * Returns: (transfer full) (element-type gchar* GArrowDataType):
+ *   The column name and value type mapping of the options.
+ *
+ * Since: 0.12.0
+ */
+GHashTable *
+garrow_csv_read_options_get_column_types(GArrowCSVReadOptions *options)
+{
+  auto priv = GARROW_CSV_READ_OPTIONS_GET_PRIVATE(options);
+  GHashTable *types = g_hash_table_new_full(g_str_hash,
+g_str_equal,
+g_free,
+g_object_unref);
+  for (const auto iter : priv->convert_options.column_types) {
+auto arrow_name = iter.first;
+auto arrow_data_type = iter.second;
+g_hash_table_insert(types,
+g_strdup(arrow_name.c_str()),
+garrow_data_type_new_raw(_data_type));
+  }
+  return types;
+}
+
 
 typedef struct GArrowCSVReaderPrivate_ {
   std::shared_ptr reader;
diff --git a/c_glib/arrow-glib/reader.h b/c_glib/arrow-glib/reader.h
index d1a3947..de33a79 100644
--- a/c_glib/arrow-glib/reader.h
+++ b/c_glib/arrow-glib/reader.h
@@ -255,6 +255,15 @@ struct _GArrowCSVReadOptionsClass
 };
 
 GArrowCSVReadOptions *garrow_csv_read_options_new(void);
+void
+garrow_csv_read_options_add_column_type(GArrowCSVReadOptions *options,
+const gchar *name,
+GArrowDataType *data_type);
+void
+garrow_csv_read_options_add_schema(GArrowCSVReadOptions *options,
+   GArrowSchema *schema);
+GHashTable *
+garrow_csv_read_options_get_column_types(GArrowCSVReadOptions *options);
 
 #define GARROW_TYPE_CSV_READER (garrow_csv_reader_get_type())
 G_DECLARE_DERIVABLE_TYPE(GArrowCSVReader,
diff --git a/c_glib/test/test-csv-reader.rb b/c_glib/test/test-csv-reader.rb
index 12897a8..3cae103 100644
--- a/c_glib/test/test-csv-reader.rb
+++ b/c_glib/test/test-csv-reader.rb
@@ -40,20 +40,56 @@ message,count
table.read)
 end
 
-def test_options
-  options = Arrow::CSVReadOptions.new
-  options.quoted = false
-  table = Arrow::CSVReader.new(open_input(<<-CSV), options)
-message,count
-"Start",2
-"Shutdown",9
-  CSV
-  columns = {
-"message

[arrow] branch master updated: ARROW-3748: [GLib] Add GArrowCSVReader

2018-11-10 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 455cde1  ARROW-3748: [GLib] Add GArrowCSVReader
455cde1 is described below

commit 455cde1096d3436fbaf3f8fbfe39503a3b1c9326
Author: Kouhei Sutou 
AuthorDate: Sun Nov 11 11:20:23 2018 +0900

ARROW-3748: [GLib] Add GArrowCSVReader

Author: Kouhei Sutou 

Closes #2934 from kou/glib-csv and squashes the following commits:

e6b70787   Add GArrowCSVReader
---
 c_glib/arrow-glib/reader.cpp  | 566 +-
 c_glib/arrow-glib/reader.h|  31 ++
 c_glib/arrow-glib/reader.hpp  |   6 +
 c_glib/doc/arrow-glib/arrow-glib-docs.xml |   4 +
 c_glib/test/test-csv-reader.rb|  59 
 5 files changed, 657 insertions(+), 9 deletions(-)

diff --git a/c_glib/arrow-glib/reader.cpp b/c_glib/arrow-glib/reader.cpp
index 296b911..5253a45 100644
--- a/c_glib/arrow-glib/reader.cpp
+++ b/c_glib/arrow-glib/reader.cpp
@@ -51,6 +51,9 @@ G_BEGIN_DECLS
  *
  * #GArrowFeatherFileReader is a class for reading columns in Feather
  * file format from input.
+ *
+ * #GArrowCSVReader is a class for reading table in CSV format from
+ * input.
  */
 
 typedef struct GArrowRecordBatchReaderPrivate_ {
@@ -888,6 +891,541 @@ 
garrow_feather_file_reader_read_names(GArrowFeatherFileReader *reader,
   }
 }
 
+
+typedef struct GArrowCSVReadOptionsPrivate_ {
+  arrow::MemoryPool *pool;
+  arrow::csv::ReadOptions read_options;
+  arrow::csv::ParseOptions parse_options;
+  arrow::csv::ConvertOptions convert_options;
+} GArrowCSVReadOptionsPrivate;
+
+enum {
+  PROP_POOL = 1,
+  PROP_USE_THREADS,
+  PROP_BLOCK_SIZE,
+  PROP_DELIMITER,
+  PROP_IS_QUOTED,
+  PROP_QUOTE_CHARACTER,
+  PROP_IS_DOUBLE_QUOTED,
+  PROP_IS_ESCAPED,
+  PROP_ESCAPE_CHARACTER,
+  PROP_ALLOW_NEWLINES_IN_VALUES,
+  PROP_IGNORE_EMPTY_LINES,
+  PROP_N_HEADER_ROWS,
+  PROP_CHECK_UTF8
+};
+
+G_DEFINE_TYPE_WITH_PRIVATE(GArrowCSVReadOptions,
+   garrow_csv_read_options,
+   G_TYPE_OBJECT)
+
+#define GARROW_CSV_READ_OPTIONS_GET_PRIVATE(object) \
+  static_cast(   \
+garrow_csv_read_options_get_instance_private(   \
+  GARROW_CSV_READ_OPTIONS(object)))
+
+static void
+garrow_csv_read_options_set_property(GObject *object,
+ guint prop_id,
+ const GValue *value,
+ GParamSpec *pspec)
+{
+  auto priv = GARROW_CSV_READ_OPTIONS_GET_PRIVATE(object);
+
+  switch (prop_id) {
+  case PROP_POOL:
+priv->pool = static_cast(g_value_get_pointer(value));
+break;
+  case PROP_USE_THREADS:
+priv->read_options.use_threads = g_value_get_boolean(value);
+break;
+  case PROP_BLOCK_SIZE:
+priv->read_options.block_size = g_value_get_int(value);
+break;
+  case PROP_DELIMITER:
+priv->parse_options.delimiter = g_value_get_schar(value);
+break;
+  case PROP_IS_QUOTED:
+priv->parse_options.quoting = g_value_get_boolean(value);
+break;
+  case PROP_QUOTE_CHARACTER:
+priv->parse_options.quote_char = g_value_get_schar(value);
+break;
+  case PROP_IS_DOUBLE_QUOTED:
+priv->parse_options.double_quote = g_value_get_boolean(value);
+break;
+  case PROP_IS_ESCAPED:
+priv->parse_options.escaping = g_value_get_boolean(value);
+break;
+  case PROP_ESCAPE_CHARACTER:
+priv->parse_options.escape_char = g_value_get_schar(value);
+break;
+  case PROP_ALLOW_NEWLINES_IN_VALUES:
+priv->parse_options.newlines_in_values = g_value_get_boolean(value);
+break;
+  case PROP_IGNORE_EMPTY_LINES:
+priv->parse_options.ignore_empty_lines = g_value_get_boolean(value);
+break;
+  case PROP_N_HEADER_ROWS:
+priv->parse_options.header_rows = g_value_get_uint(value);
+break;
+  case PROP_CHECK_UTF8:
+priv->convert_options.check_utf8 = g_value_get_boolean(value);
+break;
+  default:
+G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec);
+break;
+  }
+}
+
+static void
+garrow_csv_read_options_get_property(GObject *object,
+ guint prop_id,
+ GValue *value,
+ GParamSpec *pspec)
+{
+  auto priv = GARROW_CSV_READ_OPTIONS_GET_PRIVATE(object);
+
+  switch (prop_id) {
+  case PROP_USE_THREADS:
+g_value_set_boolean(value, priv->read_options.use_threads);
+break;
+  case PROP_BLOCK_SIZE:
+g_value_set_int(value, priv->read_options.block_size);
+break;
+  case PROP_DELIMITER:
+g_value_set_schar(value, priv->parse_options.delimiter);
+break;
+  case PROP_IS_QUOTED:
+g_value_set_boolean(value, priv->parse_options.quoting);
+break;
+  case PROP_

[arrow] branch master updated: ARROW-3749: [GLib] Fix typos

2018-11-10 Thread shiro

This is an automated email from the ASF dual-hosted git repository.

shiro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 2534cd7  ARROW-3749: [GLib] Fix typos
2534cd7 is described below

commit 2534cd7830c80359ed36dd0ad2352ac00d4a4370
Author: Kouhei Sutou 
AuthorDate: Sat Nov 10 23:34:32 2018 +0900

ARROW-3749: [GLib] Fix typos

Author: Kouhei Sutou 

Closes #2935 from kou/glib-typo and squashes the following commits:

92f6dc7c   Fix typos
---
 c_glib/arrow-glib/error.h| 2 +-
 c_glib/test/test-record-batch.rb | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/c_glib/arrow-glib/error.h b/c_glib/arrow-glib/error.h
index f195e7d..98eae16 100644
--- a/c_glib/arrow-glib/error.h
+++ b/c_glib/arrow-glib/error.h
@@ -39,7 +39,7 @@ G_BEGIN_DECLS
  * @GARROW_ERROR_PLASMA_OBJECT_NONEXISTENT: Object doesn't exist on Plasma.
  * @GARROW_ERROR_PLASMA_STORE_FULL: Store full error on Plasma.
  * @GARROW_ERROR_PLASMA_OBJECT_ALREADY_SEALED: Object already sealed on Plasma.
- * @GARROW_ERROR_CODE_GENRATION: Error generating code for expression 
evaluation
+ * @GARROW_ERROR_CODE_GENERATION: Error generating code for expression 
evaluation
  *   in Gandiva.
  * @GARROW_ERROR_ARROW: Error in Gandiva-Arrow integration.
  * @GARROW_ERROR_EXPRESSION_VALIDATION: Validation errors in expression given 
for code generation.
diff --git a/c_glib/test/test-record-batch.rb b/c_glib/test/test-record-batch.rb
index 1cbc7cb..23078d7 100644
--- a/c_glib/test/test-record-batch.rb
+++ b/c_glib/test/test-record-batch.rb
@@ -15,7 +15,7 @@
 # specific language governing permissions and limitations
 # under the License.
 
-class TestTable < Test::Unit::TestCase
+class TestRecordBatch < Test::Unit::TestCase
   include Helper::Buildable
 
   sub_test_case(".new") do

62 matches

Mail list logo