This is an automated email from the ASF dual-hosted git repository.
ianmcook pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-experiments.git
The following commit(s) were added to refs/heads/main by this push:
new 260ca77 [http] Add range request examples (#43)
260ca77 is described below
commit 260ca77eef1d0ce6195e4b67d31739a514e6235f
Author: Ian Cook <[email protected]>
AuthorDate: Sun Dec 1 22:48:43 2024 -0500
[http] Add range request examples (#43)
* Add range request examples
* Use '.part' file extension
* Improve JS server readme
* Improve JS server
* Improve examples
* Improve server example
---
http/get_range/README.md | 2 +-
http/get_range/curl/.gitignore | 19 ++++++++++++
http/get_range/{ => curl/client}/README.md | 6 ++--
http/get_range/curl/client/client.sh | 50 ++++++++++++++++++++++++++++++
http/get_range/js/.gitignore | 21 +++++++++++++
http/get_range/js/server/README.md | 37 ++++++++++++++++++++++
http/get_range/js/server/serve.json | 11 +++++++
7 files changed, 143 insertions(+), 3 deletions(-)
diff --git a/http/get_range/README.md b/http/get_range/README.md
index 0902500..956dc46 100644
--- a/http/get_range/README.md
+++ b/http/get_range/README.md
@@ -19,4 +19,4 @@
# HTTP GET Arrow Data: Range Request Examples
-This directory contains examples of HTTP servers/clients that send/receive
data of known size (`Content-Length`) in the Arrow IPC streaming format and
support range requests (`Accept-Range: bytes`).
+This directory contains examples of HTTP servers/clients that send/receive
data of known size (`Content-Length`) in the Arrow IPC streaming format and
support range requests (`Accept-Ranges: bytes`).
diff --git a/http/get_range/curl/.gitignore b/http/get_range/curl/.gitignore
new file mode 100644
index 0000000..f00067f
--- /dev/null
+++ b/http/get_range/curl/.gitignore
@@ -0,0 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+*.arrows
+*.arrows.part*
diff --git a/http/get_range/README.md b/http/get_range/curl/client/README.md
similarity index 71%
copy from http/get_range/README.md
copy to http/get_range/curl/client/README.md
index 0902500..33fd7e7 100644
--- a/http/get_range/README.md
+++ b/http/get_range/curl/client/README.md
@@ -17,6 +17,8 @@
under the License.
-->
-# HTTP GET Arrow Data: Range Request Examples
+# HTTP GET Arrow Data: Range Request curl Client Example
-This directory contains examples of HTTP servers/clients that send/receive
data of known size (`Content-Length`) in the Arrow IPC streaming format and
support range requests (`Accept-Range: bytes`).
+This directory contains examples of `curl` commands that send HTTP GET
requests with the `Range` request header.
+
+To run this example, first start one of the range request server examples in
the parent directory, then run the shell commands in `client.sh`.
diff --git a/http/get_range/curl/client/client.sh
b/http/get_range/curl/client/client.sh
new file mode 100644
index 0000000..e11bc8e
--- /dev/null
+++ b/http/get_range/curl/client/client.sh
@@ -0,0 +1,50 @@
+#!/bin/sh
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+### Use range requests to download an Arrow IPC stream file in two parts
+
+# Get the length of the file `random.arrows` in bytes
+curl -I localhost:8008/random.arrows
+# Content-Length: 13550776
+
+# Download the first half of the file to `random.arrows.part1`
+curl -r 0-6775388 localhost:8008/random.arrows -o random.arrows.part1
+
+# Download the second half of the file to `random.arrows.part2`
+curl -r 6775389-13550776 localhost:8008/random.arrows -o random.arrows.part2
+
+# Combine the two separate files into one file `random.arrows` then delete them
+cat random.arrows.part1 random.arrows.part2 > random.arrows
+rm random.arrows.part1 random.arrows.part2
+
+# Clean up
+rm random.arrows
+
+
+### Simulate an interrupted download over a slow connection
+
+# Begin downloading the file at 1M/s but interrupt after five seconds
+timeout 5s curl --limit-rate 1M localhost:8008/random.arrows -o random.arrows
+
+# Resume the download at 1M/s
+curl -C - --limit-rate 1M localhost:8008/random.arrows -o random.arrows
+
+# Clean up
+rm random.arrows
diff --git a/http/get_range/js/.gitignore b/http/get_range/js/.gitignore
new file mode 100644
index 0000000..8e6bd3e
--- /dev/null
+++ b/http/get_range/js/.gitignore
@@ -0,0 +1,21 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+/**/node_modules
+package-lock.json
+package.json
+*.arrows
diff --git a/http/get_range/js/server/README.md
b/http/get_range/js/server/README.md
new file mode 100644
index 0000000..6d03159
--- /dev/null
+++ b/http/get_range/js/server/README.md
@@ -0,0 +1,37 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# HTTP GET Arrow Data: Range Request Node.js Server Example
+
+The example in this directory shows how to use the Node.js package
[`serve`](https://www.npmjs.com/package/serve) (which supports range requests)
to serve a static Arrow IPC stream file over HTTP.
+
+To run this example, copy the file `random.arrows` from the directory
`data/rand-many-types/` into the current directory:
+
+```sh
+cp ../../../../data/rand-many-types/random.arrows .
+```
+
+Then start the HTTP server to serve this file:
+
+```sh
+npx serve -l 8008
+```
+
+> [!NOTE]
+> The npm package `serve` _should_ automatically set the `Content-Type` header
to `application/vnd.apache.arrow.stream` when serving a file with extension
`.arrows`, because [the Arrow IPC stream format is officially registered with
IANA](https://www.iana.org/assignments/media-types/application/vnd.apache.arrow.stream)
and most web servers including `serve` use registration data from IANA to
determine the media type of a file based on its file extension and set the
`Content-Type` header [...]
diff --git a/http/get_range/js/server/serve.json
b/http/get_range/js/server/serve.json
new file mode 100644
index 0000000..e3016d8
--- /dev/null
+++ b/http/get_range/js/server/serve.json
@@ -0,0 +1,11 @@
+{
+ "headers": [
+ {
+ "source" : "**/*.arrows",
+ "headers" : [{
+ "key" : "Content-Type",
+ "value" : "application/vnd.apache.arrow.stream"
+ }]
+ }
+ ]
+}
