Hi folks This patch for mod_http_upload_external does two things:
1. It adds a new config option `mod_http_upload_external_quota` which is a number representing bytes. The quota is included as a "q" parameter in the URL of the PUT request to the external service. This allows the external service to enforce quotas, either globally or per user. 2. New config options `mod_http_upload_external_include_jid_hash` and `mod_http_upload_external_jid_hash_salt` The first is a boolean which makes it possible to group files per hashed/salted JID. The second is the salt, so that only admins can figure out the hash for a particular JID. This option makes it possible to remove all files for a particular JID (useful for GDPR compliance) and also enables the external service to enforce per-user quotas. I've made a Pull Request to xmpp-http-upload.py to do just that. https://github.com/horazont/xmpp-http-upload/pull/9 --- Feedback and comments welcome. If no-one objects, I'll push to the prosody-modules repo. Thanks JC -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
diff -rub
../prosody-modules/mod_http_upload_external/mod_http_upload_external.lua
mod_http_upload_external/mod_http_upload_external.lua
--- ../prosody-modules/mod_http_upload_external/mod_http_upload_external.lua
2019-01-12 09:55:28.000000000 +0000
+++ mod_http_upload_external/mod_http_upload_external.lua 2019-01-13
10:06:30.000000000 +0000
@@ -11,9 +11,14 @@
local http = require "util.http";
local dataform = require "util.dataforms".new;
local HMAC = require "util.hashes".hmac_sha256;
+local sha256 = require "util.hashes".sha256;
+local jid_bare = require "util.jid".bare;
-- config
+local quota = module:get_option_number(module.name .. "_quota", 0);
local file_size_limit = module:get_option_number(module.name ..
"_file_size_limit", 100 * 1024 * 1024); -- 100 MB
+local include_jid_hash = module:get_option_boolean(module.name ..
"_include_jid_hash", false);
+local jid_hash_salt = module:get_option_string(module.name ..
"_jid_hash_salt", '');
local base_url = assert(module:get_option_string(module.name .. "_base_url"),
module.name .. "_base_url is a required option");
local secret = assert(module:get_option_string(module.name .. "_secret"),
@@ -43,16 +48,16 @@
{ name = "max-file-size", type = "text-single" },
}:form({ ["max-file-size"] = tostring(file_size_limit) }, "result"));
-local function magic_crypto_dust(random, filename, filesize, filetype)
+local function magic_crypto_dust(jid_hash, random, filename, filesize,
filetype)
local param, message;
if token_protocol == "v1" then
- param, message = "v", string.format("%s/%s %d", random,
filename, filesize);
+ param, message = "v", string.format("%s%s/%s %d", jid_hash,
random, filename, filesize);
else
- param, message = "v2", string.format("%s/%s\0%d\0%s", random,
filename, filesize, filetype);
+ param, message = "v2", string.format("%s%s/%s\0%d\0%s",
jid_hash, random, filename, filesize, filetype);
end
local digest = HMAC(secret, message, true);
random, filename = http.urlencode(random), http.urlencode(filename);
- return base_url .. random .. "/" .. filename, "?"..param.."=" .. digest;
+ return base_url .. jid_hash .. random .. "/" .. filename,
"?"..param.."=" .. digest .. "&q=" .. quota;
end
local function handle_request(origin, stanza, xmlns, filename, filesize,
filetype)
@@ -80,7 +85,8 @@
return nil, nil;
end
local random = uuid();
- local get_url, verify = magic_crypto_dust(random, filename, filesize,
filetype);
+ local jid_hash = include_jid_hash and
(sha256(jid_bare(stanza.attr.from) .. jid_hash_salt, true) .. '/') or '';
+ local get_url, verify = magic_crypto_dust(jid_hash, random, filename,
filesize, filetype);
local put_url = get_url .. verify;
module:log("debug", "Handing out upload slot %s to %s@%s [%d %s]",
get_url, origin.username, origin.host, filesize, filetype);
Only in ../prosody-modules/mod_http_upload_external:
mod_http_upload_external.lua~
Only in mod_http_upload_external: .mod_http_upload_external.lua.swp
diff -rub ../prosody-modules/mod_http_upload_external/README.markdown
mod_http_upload_external/README.markdown
--- ../prosody-modules/mod_http_upload_external/README.markdown 2019-01-12
07:52:21.000000000 +0000
+++ mod_http_upload_external/README.markdown 2019-01-13 10:56:19.000000000
+0000
@@ -3,8 +3,7 @@
labels: 'Stage-Alpha'
---
-Introduction
-============
+# Introduction
This module implements [XEP-0363], which lets clients upload files
over HTTP to an external web server.
@@ -12,8 +11,7 @@
This module generates URLs that are signed using a HMAC. Any web service that
can authenticate
these URLs can be used.
-Implementations
----------------
+## Implementations
* [PHP
implementation](https://hg.prosody.im/prosody-modules/raw-file/tip/mod_http_upload_external/share.php)
* [Python3+Flask implementation](https://github.com/horazont/xmpp-http-upload)
@@ -23,14 +21,12 @@
To implement your own service compatible with this module, check out the
implementation notes below
(and if you publish your implementation - let us know!).
-Configuration
-=============
+## Configuration
Add `"http_upload_external"` to modules_enabled in your global section, or
under the host(s) you wish
to use it on.
-External URL
-------------
+### External URL
You need to provide the path to the external service. Ensure it ends with '/'.
@@ -40,8 +36,7 @@
http_upload_external_base_url = "https://your.example.com/path/to/share.php/";
```
-Secret
-------
+### Secret
Set a long and unpredictable string as your secret. This is so the upload
service can verify that
the upload comes from mod_http_upload_external, and random strangers can't
upload to your server.
@@ -52,8 +47,7 @@
You need to set exactly the same secret string in your external service.
-Limits
-------
+### Limits
A maximum file size can be set by:
@@ -63,13 +57,47 @@
Default is 100MB (100\*1024\*1024).
-Compatibility
-=============
+## Include hashed JID in URL
+
+You might want to have a unique URL per user JID.
+
+We do this by prepending the URL with the hashed and salted JID.
+
+This can help with compliance requirements, by making it easier for admins (but
+no-one else) to identify files uploaded by a particular user.
+
+It also provides necessary information for the external service to apply
per-user
+quotas.
+
+This requires however that the external service uses the same
+filesystem path as given in the URL.
+
+``` {.lua}
+http_upload_external_include_jid_hash = true;
+http_upload_external_jid_hash_salt = 'uugahxaiy3peezei1tufeiLeyae8Op4A'; --
Replace with your own salt value!
+```
+
+### User quota
+
+It's possible to set a file upload quota which will be communicated to the
+external service.
+
+If you're including the hashed JID in the URL, then the external service has
+enough information to set per-user quotas (it's up to you to check whether it
+actually does).
+
+Otherwise the quota could be used for all uploaded files across all users.
+
+
+``` {.lua}
+http_upload_external_quota = 100*1024*1024; -- 100 Megabytes
+```
+
+## Compatibility
Works with Prosody 0.9.x and later.
-Implementation
-==============
+## Implementation
To implement your own external service that is compatible with this module,
you need to expose a
simple API that allows the HTTP GET, HEAD and PUT methods on arbitrary URLs
located on your service.
signature.asc
Description: PGP signature
