This is an automated email from the ASF dual-hosted git repository.

bamaer pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hop.git


The following commit(s) were added to refs/heads/main by this push:
     new 414b851325 Update the VFS supported operations matrix and document 
append on S3, Azure and Google Cloud Storage, fixes #9380 (#9384)
414b851325 is described below

commit 414b851325e3435f10798adc885fa2a6b14bc0cf
Author: Adalennis <[email protected]>
AuthorDate: Sun Nov 8 08:45:58 2026 +0100

    Update the VFS supported operations matrix and document append on S3, Azure 
and Google Cloud Storage, fixes #9380 (#9384)
---
 docs/hop-user-manual/modules/ROOT/pages/vfs.adoc            | 13 ++++++++++---
 docs/hop-user-manual/modules/ROOT/pages/vfs/aws-s3-vfs.adoc |  7 +++++++
 .../modules/ROOT/pages/vfs/azure-blob-storage-vfs.adoc      |  7 +++++++
 .../modules/ROOT/pages/vfs/google-cloud-storage-vfs.adoc    |  7 +++++++
 4 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/docs/hop-user-manual/modules/ROOT/pages/vfs.adoc 
b/docs/hop-user-manual/modules/ROOT/pages/vfs.adoc
index e17ef4f318..e7f49ce566 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/vfs.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/vfs.adoc
@@ -321,12 +321,16 @@ Server-side restrictions (read-only mounts, bucket 
policies, account permissions
 |Scheme|Read|Write|Append|List|Create/Delete|Rename|Random read|Random 
write|Last-modified
 
 // Hop-managed providers
-|`s3`               |✓|✓|✗|✓|✓|✓|✓|✗|read
-|`azure`            |✓|✓|✗|✓|✓|✓|✗|✗|read
-|`gs`               |✓|✓|✗|✓|✓|✗|✗|✗|read/set
+|`s3`               |✓|✓|✓|✓|✓|✓|✓|✗|read
+|`azure`            |✓|✓|✓|✓|✓|✓|✓|✗|read
+|`gs`               |✓|✓|✓|✓|✓|✗|✓|✗|read/set
 |`googledrive`      |✓|✓|✗|✓|✓|✓|✓|✗|read
 |`dropbox`          |✓|✓|✗|✓|✓|✓|✗|✗|read
 |`+<minio-conn>+`   |✓|✓|✗|✓|✓|✓|✓|✗|read
+|`+<databricks-conn>+` |✓|✓|✗|✓|✓|✓|✗|✗|read
+|`+<git-conn>+` (see below) |✓|✓|✓|✓|✓|✓|✗|✗|read
+|`+<hdfs-conn>+`    |✓|✓|✗|✓|✓|✓|✗|✗|read
+|`+<smb-conn>+`     |✓|✓|✓|✓|✓|✓|✓|✓|read
 |`webdav4`, `webdav4s`, named WebDAV connection |✓|✓|✗|✓|✓|✓|✓|✗|read
 
 // Apache VFS providers
@@ -340,3 +344,6 @@ Server-side restrictions (read-only mounts, bucket 
policies, account permissions
 |`ftp`, `ftps`      |✓|✓|✓|✓|✓|✓|✓|✗|read
 |`sftp`             |✓|✓|✓|✓|✓|✓|✓|✗|read/set
 |===
+
+A Git connection reads files although it doesn't declare the read capability.
+A read-only Git connection, the default, refuses writes.
diff --git a/docs/hop-user-manual/modules/ROOT/pages/vfs/aws-s3-vfs.adoc 
b/docs/hop-user-manual/modules/ROOT/pages/vfs/aws-s3-vfs.adoc
index f9bfa89244..f7844a839d 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/vfs/aws-s3-vfs.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/vfs/aws-s3-vfs.adoc
@@ -104,6 +104,13 @@ One connection to S3 is often limited to a few MB/s, so 
reading many blocks at t
 Text File Input running in parallel keeps up to copies × read-ahead blocks 
requests in flight: 20 copies with 4 blocks each need 80 connections.
 With fewer connections allowed than that, the other requests wait for a free 
one.
 
+== Appending to a file
+
+Options such as *Append* in the Text file output transform add to an existing 
object.
+S3 has no append operation, so Hop rebuilds the object with a multipart 
upload: the existing content is copied on the server, and only the new bytes 
are uploaded.
+An object smaller than 5 MiB is read back and uploaded again with the new 
bytes.
+The existing content is only copied or read when the object hasn't changed 
since Hop opened it for the append, otherwise the append fails.
+
 == Usage and testing
 
 To test if the configuration works you can simply upload a small CSV file in 
an S3 bucket and then use File/Open in Hop GUI.
diff --git 
a/docs/hop-user-manual/modules/ROOT/pages/vfs/azure-blob-storage-vfs.adoc 
b/docs/hop-user-manual/modules/ROOT/pages/vfs/azure-blob-storage-vfs.adoc
index 2ac09700b8..89bde3c580 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/vfs/azure-blob-storage-vfs.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/vfs/azure-blob-storage-vfs.adoc
@@ -115,6 +115,13 @@ NOTE: A signature stops working once it expires, and Hop 
reports the resulting f
 authentication error. Renew the token, or use `Managed Identity` where you 
want credentials that
 rotate for you.
 
+== Appending to a file
+
+Options such as *Append* in the Text file output transform add to an existing 
file.
+Hop adds the new bytes as extra blocks of the blob and commits them after the 
existing ones, so the existing content is not uploaded again.
+A blob that was uploaded in a single request has no blocks yet, so its content 
is uploaded again with the new bytes the first time.
+If the file changes while the append runs, the append fails instead of losing 
data.
+
 == Usage and testing
 
 To test if the configuration works you can simply upload a small CSV file in 
an Azure Storage folder and then use File/Open in Hop GUI.
diff --git 
a/docs/hop-user-manual/modules/ROOT/pages/vfs/google-cloud-storage-vfs.adoc 
b/docs/hop-user-manual/modules/ROOT/pages/vfs/google-cloud-storage-vfs.adoc
index a36f46a495..6d953b49ee 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/vfs/google-cloud-storage-vfs.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/vfs/google-cloud-storage-vfs.adoc
@@ -97,6 +97,13 @@ A write is never ended.
 
 These messages are written to the general Hop log with the prefix `Google 
Cloud Storage:`, for example in the console output of `hop-run`.
 
+== Appending to a file
+
+Options such as *Append* in the Text file output transform add to an existing 
object.
+Google Cloud Storage objects can't be changed, so Hop writes the new bytes to 
a temporary object, joins it to the end of the existing object with a compose 
when the file is closed, and deletes the temporary object.
+If the object changes while the append runs, the append fails instead of 
losing data.
+A composed object has no MD5 hash, only a CRC32C checksum.
+
 == Usage and testing
 
 To test if the configuration works type use File/Open in the GUI and type in 
gs:// as a file location.

Reply via email to