Hello community, here is the log from the commit of package s3backer for openSUSE:Factory checked in at 2017-02-02 15:46:00 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/s3backer (Old) and /work/SRC/openSUSE:Factory/.s3backer.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "s3backer" Changes: -------- --- /work/SRC/openSUSE:Factory/s3backer/s3backer.changes 2016-07-27 16:13:19.000000000 +0200 +++ /work/SRC/openSUSE:Factory/.s3backer.new/s3backer.changes 2017-02-03 20:01:07.611978940 +0100 @@ -1,0 +2,8 @@ +Wed Feb 1 23:11:25 UTC 2017 - [email protected] + +- Update to release 1.4.4 + + Added `--defaultContentEncoding' for non-compliant backends (issue #68) + + Fixed auth bug when prefix contains URL-encodable char (issue #69) + + Remove restriction preventing streaming encryption modes (issue #70) + +------------------------------------------------------------------- Old: ---- s3backer-1.4.3.tar.gz New: ---- s3backer-1.4.4.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ s3backer.spec ++++++ --- /var/tmp/diff_new_pack.XroOXR/_old 2017-02-03 20:01:08.275885493 +0100 +++ /var/tmp/diff_new_pack.XroOXR/_new 2017-02-03 20:01:08.279884930 +0100 @@ -1,7 +1,7 @@ # # spec file for package s3backer # -# Copyright (c) 2016 SUSE LINUX GmbH, Nuernberg, Germany. +# Copyright (c) 2017 SUSE LINUX GmbH, Nuernberg, Germany. # Copyright 2008 Archie L. Cobbs. # # All modifications and additions to the file contributed by third parties @@ -18,7 +18,7 @@ Name: s3backer -Version: 1.4.3 +Version: 1.4.4 Release: 0 Summary: FUSE-based single file backing store via Amazon S3 License: GPL-2.0+ ++++++ s3backer-1.4.3.tar.gz -> s3backer-1.4.4.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/s3backer-1.4.3/CHANGES new/s3backer-1.4.4/CHANGES --- old/s3backer-1.4.3/CHANGES 2016-07-25 20:55:00.000000000 +0200 +++ new/s3backer-1.4.4/CHANGES 2017-02-02 00:07:49.000000000 +0100 @@ -1,3 +1,9 @@ +Version 1.4.4 released February 1, 2017 + + - Added `--defaultContentEncoding' for non-compliant backends (issue #68) + - Fixed auth bug when prefix contains URL-encodable char (issue #69) + - Remove restriction preventing streaming encryption modes (issue #70) + Version 1.4.3 released July 25, 2016 - Add support for STANDARD_IA storage class (issue #59) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/s3backer-1.4.3/configure new/s3backer-1.4.4/configure --- old/s3backer-1.4.3/configure 2016-07-25 20:55:36.000000000 +0200 +++ new/s3backer-1.4.4/configure 2017-02-02 00:08:34.000000000 +0100 @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for s3backer FUSE filesystem backed by Amazon S3 1.4.3. +# Generated by GNU Autoconf 2.69 for s3backer FUSE filesystem backed by Amazon S3 1.4.4. # # Report bugs to <https://github.com/archiecobbs/s3backer>. # @@ -580,8 +580,8 @@ # Identity of this package. PACKAGE_NAME='s3backer FUSE filesystem backed by Amazon S3' PACKAGE_TARNAME='s3backer' -PACKAGE_VERSION='1.4.3' -PACKAGE_STRING='s3backer FUSE filesystem backed by Amazon S3 1.4.3' +PACKAGE_VERSION='1.4.4' +PACKAGE_STRING='s3backer FUSE filesystem backed by Amazon S3 1.4.4' PACKAGE_BUGREPORT='https://github.com/archiecobbs/s3backer' PACKAGE_URL='' @@ -1279,7 +1279,7 @@ # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures s3backer FUSE filesystem backed by Amazon S3 1.4.3 to adapt to many kinds of systems. +\`configure' configures s3backer FUSE filesystem backed by Amazon S3 1.4.4 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1345,7 +1345,7 @@ if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of s3backer FUSE filesystem backed by Amazon S3 1.4.3:";; + short | recursive ) echo "Configuration of s3backer FUSE filesystem backed by Amazon S3 1.4.4:";; esac cat <<\_ACEOF @@ -1446,7 +1446,7 @@ test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -s3backer FUSE filesystem backed by Amazon S3 configure 1.4.3 +s3backer FUSE filesystem backed by Amazon S3 configure 1.4.4 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. @@ -1794,7 +1794,7 @@ This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by s3backer FUSE filesystem backed by Amazon S3 $as_me 1.4.3, which was +It was created by s3backer FUSE filesystem backed by Amazon S3 $as_me 1.4.4, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ @@ -2657,7 +2657,7 @@ # Define the identity of the package. PACKAGE='s3backer' - VERSION='1.4.3' + VERSION='1.4.4' cat >>confdefs.h <<_ACEOF @@ -5386,7 +5386,7 @@ # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by s3backer FUSE filesystem backed by Amazon S3 $as_me 1.4.3, which was +This file was extended by s3backer FUSE filesystem backed by Amazon S3 $as_me 1.4.4, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -5452,7 +5452,7 @@ cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -s3backer FUSE filesystem backed by Amazon S3 config.status 1.4.3 +s3backer FUSE filesystem backed by Amazon S3 config.status 1.4.4 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/s3backer-1.4.3/configure.ac new/s3backer-1.4.4/configure.ac --- old/s3backer-1.4.3/configure.ac 2016-07-25 20:55:00.000000000 +0200 +++ new/s3backer-1.4.4/configure.ac 2017-02-02 00:07:49.000000000 +0100 @@ -32,7 +32,7 @@ # this exception statement from all source files in the program, then # also delete it here. -AC_INIT([s3backer FUSE filesystem backed by Amazon S3], [1.4.3], [https://github.com/archiecobbs/s3backer], [s3backer]) +AC_INIT([s3backer FUSE filesystem backed by Amazon S3], [1.4.4], [https://github.com/archiecobbs/s3backer], [s3backer]) AC_CONFIG_AUX_DIR(scripts) AM_INIT_AUTOMAKE(foreign) dnl AM_MAINTAINER_MODE diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/s3backer-1.4.3/gitrev.c new/s3backer-1.4.4/gitrev.c --- old/s3backer-1.4.3/gitrev.c 2016-07-25 20:55:47.000000000 +0200 +++ new/s3backer-1.4.4/gitrev.c 2017-02-02 00:08:37.000000000 +0100 @@ -1 +1 @@ -const char *const s3backer_version = "1.4.3"; +const char *const s3backer_version = "1.4.4"; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/s3backer-1.4.3/http_io.c new/s3backer-1.4.4/http_io.c --- old/s3backer-1.4.3/http_io.c 2016-07-25 20:55:00.000000000 +0200 +++ new/s3backer-1.4.4/http_io.c 2017-01-07 20:39:17.000000000 +0100 @@ -233,6 +233,8 @@ static int http_io_add_auth(struct http_io_private *priv, struct http_io *io, time_t now, const void *payload, size_t plen); static int http_io_add_auth2(struct http_io_private *priv, struct http_io *io, time_t now, const void *payload, size_t plen); static int http_io_add_auth4(struct http_io_private *priv, struct http_io *io, time_t now, const void *payload, size_t plen); +static size_t url_encode(const char *src, size_t len, char *dst, size_t buflen, int encode_slash); +static void digest_url_encoded(EVP_MD_CTX* hash_ctx, const char *data, size_t len, int encode_slash); /* EC2 IAM thread */ static void *update_iam_credentials_main(void *arg); @@ -349,12 +351,6 @@ r = EINVAL; goto fail4; } - if (EVP_CIPHER_block_size(priv->cipher) != EVP_CIPHER_iv_length(priv->cipher)) { - (*config->log)(LOG_ERR, "invalid encryption cipher `%s': block size %d != IV length %d", - config->encryption, EVP_CIPHER_block_size(priv->cipher), EVP_CIPHER_iv_length(priv->cipher)); - r = EINVAL; - goto fail4; - } cipher_key_len = EVP_CIPHER_key_length(priv->cipher); priv->keylen = config->key_length > 0 ? config->key_length : cipher_key_len; if (priv->keylen < cipher_key_len || priv->keylen > sizeof(priv->key)) { @@ -537,6 +533,7 @@ /* List blocks */ do { + char url_encoded_prefix[strlen(config->prefix) * 3 + 1]; const time_t now = time(NULL); /* Reset XML parser state */ @@ -545,16 +542,19 @@ XML_SetElementHandler(io.xml, http_io_list_elem_start, http_io_list_elem_end); XML_SetCharacterDataHandler(io.xml, http_io_list_text); + /* URL-encode prefix */ + url_encode(config->prefix, strlen(config->prefix), url_encoded_prefix, sizeof(url_encoded_prefix), 1); + /* Format URL */ snprintf(urlbuf, sizeof(urlbuf), "%s%s?", config->baseURL, config->vhost ? "" : config->bucket); /* Add URL parameters (note: must be in "canonical query string" format for proper authentication) */ if (io.list_truncated) { snprintf(urlbuf + strlen(urlbuf), sizeof(urlbuf) - strlen(urlbuf), "%s=%s%0*jx&", - LIST_PARAM_MARKER, config->prefix, S3B_BLOCK_NUM_DIGITS, (uintmax_t)io.last_block); + LIST_PARAM_MARKER, url_encoded_prefix, S3B_BLOCK_NUM_DIGITS, (uintmax_t)io.last_block); } snprintf(urlbuf + strlen(urlbuf), sizeof(urlbuf) - strlen(urlbuf), "%s=%u", LIST_PARAM_MAX_KEYS, LIST_BLOCKS_CHUNK); - snprintf(urlbuf + strlen(urlbuf), sizeof(urlbuf) - strlen(urlbuf), "&%s=%s", LIST_PARAM_PREFIX, config->prefix); + snprintf(urlbuf + strlen(urlbuf), sizeof(urlbuf) - strlen(urlbuf), "&%s=%s", LIST_PARAM_PREFIX, url_encoded_prefix); /* Add Date header */ http_io_add_date(priv, &io, now); @@ -1124,6 +1124,8 @@ did_read = io.buf_size - io.bufs.rdremain; /* Check Content-Encoding and decode if necessary */ + if (*io.content_encoding == '\0' && config->default_ce != NULL) + snprintf(io.content_encoding, sizeof(io.content_encoding), "%s", config->default_ce); for ( ; r == 0 && *io.content_encoding != '\0'; *layer = '\0') { /* Find next encoding layer */ @@ -2060,7 +2062,7 @@ #endif /* Canonical URI */ - EVP_DigestUpdate(hash_ctx, (const u_char *)uripath, uripath_len); + digest_url_encoded(hash_ctx, uripath, uripath_len, 0); EVP_DigestUpdate(hash_ctx, (const u_char *)"\n", 1); #if DEBUG_AUTHENTICATION snprintf(sigbuf + strlen(sigbuf), sizeof(sigbuf) - strlen(sigbuf), "%.*s\n", (int)uripath_len, uripath); @@ -2242,6 +2244,44 @@ } /* + * Add data to digest, but in URL-encoded form. + */ +static void +digest_url_encoded(EVP_MD_CTX* hash_ctx, const char *data, size_t len, int encode_slash) +{ + char buf[len * 3 + 1]; + + len = url_encode(data, len, buf, sizeof(buf), encode_slash); + EVP_DigestUpdate(hash_ctx, (const u_char *)buf, len); +} + +/* + * URL-encode the given input. + */ +static size_t +url_encode(const char *src, size_t len, char *dst, size_t buflen, int encode_slash) +{ + char *const dst_base = dst; + size_t elen; + + while (len-- > 0) { + const char ch = *src++; + if (isalnum(ch) || ch == '_' || ch == '-' || ch == '~' || ch == '.' || (ch == '/' && !encode_slash)) { + snprintf(dst, buflen, "%c", ch); + elen = 1; + } else { + snprintf(dst, buflen, "%%%02X", (int)ch & 0xff); + elen = 3; + } + dst += elen; + buflen -= elen; + } + if (buflen > 0) + *dst = '\0'; + return dst - dst_base; +} + +/* * Create URL for a block, and return pointer to the URL's URI path. */ static void diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/s3backer-1.4.3/http_io.h new/s3backer-1.4.4/http_io.h --- old/s3backer-1.4.3/http_io.h 2016-07-22 22:15:35.000000000 +0200 +++ new/s3backer-1.4.4/http_io.h 2016-11-07 17:54:15.000000000 +0100 @@ -64,6 +64,7 @@ const char *cacert; const char *password; const char *encryption; + const char *default_ce; u_int key_length; int debug; int debug_http; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/s3backer-1.4.3/s3b_config.c new/s3backer-1.4.4/s3b_config.c --- old/s3backer-1.4.3/s3b_config.c 2016-07-22 22:15:35.000000000 +0200 +++ new/s3backer-1.4.4/s3b_config.c 2016-11-07 17:54:15.000000000 +0100 @@ -365,6 +365,10 @@ .offset= offsetof(struct s3b_config, http_io.prefix), }, { + .templ= "--defaultContentEncoding=%s", + .offset= offsetof(struct s3b_config, http_io.default_ce), + }, + { .templ= "--readOnly", .offset= offsetof(struct s3b_config, fuse_ops.read_only), .value= 1 @@ -1541,6 +1545,8 @@ (*config.log)(LOG_DEBUG, "%24s: \"%s\"", "region", config.http_io.region); (*config.log)(LOG_DEBUG, "%24s: \"%s\"", config.test ? "testdir" : "bucket", config.http_io.bucket); (*config.log)(LOG_DEBUG, "%24s: \"%s\"", "prefix", config.http_io.prefix); + (*config.log)(LOG_DEBUG, "%24s: \"%s\"", "defaultContentEncoding", + config.http_io.default_ce != NULL ? config.http_io.default_ce : "(none)"); (*config.log)(LOG_DEBUG, "%24s: %s", "list_blocks", config.list_blocks ? "true" : "false"); (*config.log)(LOG_DEBUG, "%24s: \"%s\"", "mount", config.mount); (*config.log)(LOG_DEBUG, "%24s: \"%s\"", "filename", config.fuse_ops.filename); @@ -1703,6 +1709,7 @@ fprintf(stderr, "\t--%-27s %s\n", "password=PASSWORD", "Encrypt using PASSWORD"); fprintf(stderr, "\t--%-27s %s\n", "passwordFile=FILE", "Encrypt using password read from FILE"); fprintf(stderr, "\t--%-27s %s\n", "prefix=STRING", "Prefix for resource names within bucket"); + fprintf(stderr, "\t--%-27s %s\n", "defaultContentEncoding=STRING", "Default HTTP Content-Encoding if none given"); fprintf(stderr, "\t--%-27s %s\n", "quiet", "Omit progress output at startup"); fprintf(stderr, "\t--%-27s %s\n", "readAhead=NUM", "Number of blocks to read-ahead"); fprintf(stderr, "\t--%-27s %s\n", "readAheadTrigger=NUM", "# of sequentially read blocks to trigger read-ahead"); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/s3backer-1.4.3/s3backer.1 new/s3backer-1.4.4/s3backer.1 --- old/s3backer-1.4.3/s3backer.1 2016-07-22 22:15:35.000000000 +0200 +++ new/s3backer-1.4.4/s3backer.1 2017-02-02 00:07:49.000000000 +0100 @@ -495,6 +495,17 @@ FUSE flag implies this flag. .It Fl \-debug-http Enable printing of HTTP headers to standard output. +.It Fl \-defaultContentEncoding=VALUE +Use this to workaround S3 backends that fail to send back the +.Pa "Content-Encoding" +header that was sent to them by +.Nm . +If a block read response contains no +.Pa "Content-Encoding" +header, this value will be substituted. +.Pp +If you get errors complaining that the content was expected to be encrypted, try setting this to +.Pa deflate,encrypt-AES-128-CBC . .It Fl \-encrypt[=CIPHER] Enable encryption and authentication of block data. See your OpenSSL documentation for a list of supported ciphers; @@ -733,7 +744,7 @@ Specify the size (in bytes) of the backed file to be exported by the filesystem. The size may have an optional suffix 'K' for kilobytes, 'M' for megabytes, 'G' for gigabytes, 'T' for terabytes, 'E' for exabytes, 'Z' for zettabytes, or 'Y' for yottabytes. .Nm -will attempt to auto-detect the block size by reading block number zero. +will attempt to auto-detect the size of the backed file by reading block number zero. If this option is not specified, the auto-detected value will be used. If this option is specified but disagrees with the auto-detected value, .Nm diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/s3backer-1.4.3/s3backer.spec new/s3backer-1.4.4/s3backer.spec --- old/s3backer-1.4.3/s3backer.spec 2016-07-25 20:55:44.000000000 +0200 +++ new/s3backer-1.4.4/s3backer.spec 2017-02-02 00:08:37.000000000 +0100 @@ -29,7 +29,7 @@ # Name: s3backer -Version: 1.4.3 +Version: 1.4.4 Release: 1 License: GNU General Public License, Version 2 Summary: FUSE-based single file backing store via Amazon S3
