Re: Web app large uploads and downloads

[Craig vN via ozdotnet]

I haven't tried with hundreds of files uploaded in one batch, would need to
start accounting for retry if they fail. But a few files of 100mb is no
problem.

On Wed, Jan 3, 2024 at 11:14 AM Greg Keogh via ozdotnet <
ozdotnet@ozdotnet.com> wrote:

> Folks (welcome to the distant future of 2024)
>
> I'm pretty sure that there is no sensible way to provide a bulk Azure Blob
> upload facility in a web hosted app (Wasm, JS or whatever). There could be
> thousands of files with a total size up to hundreds of MB. Managers are
> currently using a WPF program I created for uploads and it feels like a
> perfectly natural process on the desktop, and it's pretty fast using
> streams on multi cores (I also optionally check for new or changed so only
> *diffs* are uploaded which often saves a lot of time).
>
> None of the components or controls I've seen are designed for huge
> uploads, and in any case, I've reported that it's technically and usability
> questionable to have "normal" users of the browser app doing this sort of
> thing. The boss of the app suite is now considering the bigger picture and
> the bulk upload feature may be delayed or moved to somewhere else in the
> flow, or the desktop program will suffice. So I'm happy the issue is
> on-hold for now.
>
> I think this is a good example of how the web browser should never have
> evolved into a host for business apps. I think the web browser, HTML, HTTP,
> REST, css and JS have diseased 21st century IT.
>
>
> *Greg K*
>
>> --
> ozdotnet mailing list
> To manage your subscription, access archives: https://codify.mailman3.com/
-- 
ozdotnet mailing list 
To manage your subscription, access archives: https://codify.mailman3.com/ 

Re: Web app large uploads and downloads

[Greg Keogh via ozdotnet]

I think it's been using azcopy for a fair while now. Azcopy has improved
dramatically over the last few years to have more switches and options. I
recently used it to bulk copy a container in one storage account to
another, then bulk upload all files with certain extensions (which wasn't
originally possible). In each case it was harder than refining fissionable
uranium, so I pasted the skeletons of the working commands into my "bag of
tricks" text file. The Blazor bulk upload is mercifully on hold while they
reconsider who should have the facility -- *Greg K*

On Wed, 3 Jan 2024 at 16:27, Dr Greg Low via ozdotnet 
wrote:

> I notice that the Azure Storage Explorer desktop app nowadays defaults to
> using azcopy under the covers to copy files. That’s a much better option
> for files of any size. Probably need a component that does that but azcopy
> has a few dependencies.
>
>
>
> It’s on GitHub as open source so you could probably check out how it does
> what it does. It’s sure fast and filesize isn’t an issue. I’ve used azcopy
> on multi-terabyte files without issue. (Apart from the hosting provider for
> the site calling to find out what was going on, given the way it worked in
> parallel and flooded their network)
>
>
>
> Regards,
>
>
>
> Greg
>
>
>
> Dr Greg Low
>
>
>
> 1300SQLSQL (1300 775 775) office | +61 419201410 mobile
>
> SQL Down Under | Web: https://sqldownunder.com
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_=DwMFAg=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3=o3oFliHztOF8D9Nbqaa7KQdqC-zkQNXWl4IqnEG58Wc=>
>  |
> About Greg:  https://about.me/greg.low
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low=DwMFAg=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3=NsAibgiqfCxsyc8m2DBKogKQcs3OqE3mkyCjmpoYxTk=>
>
>
>
> *From:* DotNet Dude via ozdotnet 
> *Sent:* Wednesday, January 3, 2024 4:17 PM
> *To:* ozDotNet 
> *Cc:* DotNet Dude 
> *Subject:* Re: Web app large uploads and downloads
>
>
>
> Yep we usually do this sort of thing with a batch process, particularly if
> there are large files. I don’t see any web app being used just to upload a
> bunch of files.
>
>
>
> For fun try asking ChatGPT or one of the others to see where they go.
>
>
>
> On Wed, 3 Jan 2024 at 11:13, Greg Keogh via ozdotnet <
> ozdotnet@ozdotnet.com> wrote:
>
> Folks (welcome to the distant future of 2024)
>
>
>
> I'm pretty sure that there is no sensible way to provide a bulk Azure Blob
> upload facility in a web hosted app (Wasm, JS or whatever). There could be
> thousands of files with a total size up to hundreds of MB. Managers are
> currently using a WPF program I created for uploads and it feels like a
> perfectly natural process on the desktop, and it's pretty fast using
> streams on multi cores (I also optionally check for new or changed so only
> *diffs* are uploaded which often saves a lot of time).
>
>
>
> None of the components or controls I've seen are designed for huge
> uploads, and in any case, I've reported that it's technically and usability
> questionable to have "normal" users of the browser app doing this sort of
> thing. The boss of the app suite is now considering the bigger picture and
> the bulk upload feature may be delayed or moved to somewhere else in the
> flow, or the desktop program will suffice. So I'm happy the issue is
> on-hold for now.
>
>
>
> I think this is a good example of how the web browser should never have
> evolved into a host for business apps. I think the web browser, HTML, HTTP,
> REST, css and JS have diseased 21st century IT.
>
>
>
> *Greg K*
>
> --
> ozdotnet mailing list
> To manage your subscription, access archives: https://codify.mailman3.com/
>
> --
> ozdotnet mailing list
> To manage your subscription, access archives: https://codify.mailman3.com/
>
-- 
ozdotnet mailing list 
To manage your subscription, access archives: https://codify.mailman3.com/ 

Re: Web app large uploads and downloads

[mike smith via ozdotnet]

It's not a direct answer, but I'd feel happier uploading to OneDrive and
passing the server a link to that.

Mike

On Sat, 30 Dec 2023, 08:29 Greg Keogh via ozdotnet, 
wrote:

> Folks (anyone working?)
>
> I've been asked to add a feature to a Blazor Webassembly app to allow
> uploads and downloads of possibly large numbers of files between the local
> file system and Blob storage. I'm not sure how to implement this feature in
> a browser hosted app.
>
> I wrote a WPF tool for "managers" which does high-performance bulk uploads
> and downloads with nice progress (the code is trivial on the desktop), but
> now they want the same feature for "normal" users in the Blazor app. Given
> how dumb and restricted browser hosted apps are, I don't know how to code
> this, or if it's even feasible.
>
> Are there some tools, techniques or tricks I can apply? Any ideas or
> suggestions anyone?
>
> Thanks,
> *Greg Keogh*
> --
> ozdotnet mailing list
> To manage your subscription, access archives: https://codify.mailman3.com/
-- 
ozdotnet mailing list 
To manage your subscription, access archives: https://codify.mailman3.com/ 

RE: Web app large uploads and downloads

[Dr Greg Low via ozdotnet]

I notice that the Azure Storage Explorer desktop app nowadays defaults to using 
azcopy under the covers to copy files. That’s a much better option for files of 
any size. Probably need a component that does that but azcopy has a few 
dependencies.

It’s on GitHub as open source so you could probably check out how it does what 
it does. It’s sure fast and filesize isn’t an issue. I’ve used azcopy on 
multi-terabyte files without issue. (Apart from the hosting provider for the 
site calling to find out what was going on, given the way it worked in parallel 
and flooded their network)

Regards,

Greg

Dr Greg Low

1300SQLSQL (1300 775 775) office | +61 419201410 mobile
SQL Down Under | Web: 
https://sqldownunder.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_=DwMFAg=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3=o3oFliHztOF8D9Nbqaa7KQdqC-zkQNXWl4IqnEG58Wc=>
 | About Greg:  
https://about.me/greg.low<https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low=DwMFAg=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3=NsAibgiqfCxsyc8m2DBKogKQcs3OqE3mkyCjmpoYxTk=>

From: DotNet Dude via ozdotnet 
Sent: Wednesday, January 3, 2024 4:17 PM
To: ozDotNet 
Cc: DotNet Dude 
Subject: Re: Web app large uploads and downloads

Yep we usually do this sort of thing with a batch process, particularly if 
there are large files. I don’t see any web app being used just to upload a 
bunch of files.

For fun try asking ChatGPT or one of the others to see where they go.

On Wed, 3 Jan 2024 at 11:13, Greg Keogh via ozdotnet 
mailto:ozdotnet@ozdotnet.com>> wrote:
Folks (welcome to the distant future of 2024)

I'm pretty sure that there is no sensible way to provide a bulk Azure Blob 
upload facility in a web hosted app (Wasm, JS or whatever). There could be 
thousands of files with a total size up to hundreds of MB. Managers are 
currently using a WPF program I created for uploads and it feels like a 
perfectly natural process on the desktop, and it's pretty fast using streams on 
multi cores (I also optionally check for new or changed so only diffs are 
uploaded which often saves a lot of time).

None of the components or controls I've seen are designed for huge uploads, and 
in any case, I've reported that it's technically and usability questionable to 
have "normal" users of the browser app doing this sort of thing. The boss of 
the app suite is now considering the bigger picture and the bulk upload feature 
may be delayed or moved to somewhere else in the flow, or the desktop program 
will suffice. So I'm happy the issue is on-hold for now.

I think this is a good example of how the web browser should never have evolved 
into a host for business apps. I think the web browser, HTML, HTTP, REST, css 
and JS have diseased 21st century IT.

Greg K
--
ozdotnet mailing list
To manage your subscription, access archives: https://codify.mailman3.com/
-- 
ozdotnet mailing list 
To manage your subscription, access archives: https://codify.mailman3.com/ 

Re: Web app large uploads and downloads

[DotNet Dude via ozdotnet]

Yep we usually do this sort of thing with a batch process, particularly if
there are large files. I don’t see any web app being used just to upload a
bunch of files.

For fun try asking ChatGPT or one of the others to see where they go.

On Wed, 3 Jan 2024 at 11:13, Greg Keogh via ozdotnet 
wrote:

> Folks (welcome to the distant future of 2024)
>
> I'm pretty sure that there is no sensible way to provide a bulk Azure Blob
> upload facility in a web hosted app (Wasm, JS or whatever). There could be
> thousands of files with a total size up to hundreds of MB. Managers are
> currently using a WPF program I created for uploads and it feels like a
> perfectly natural process on the desktop, and it's pretty fast using
> streams on multi cores (I also optionally check for new or changed so only
> *diffs* are uploaded which often saves a lot of time).
>
> None of the components or controls I've seen are designed for huge
> uploads, and in any case, I've reported that it's technically and usability
> questionable to have "normal" users of the browser app doing this sort of
> thing. The boss of the app suite is now considering the bigger picture and
> the bulk upload feature may be delayed or moved to somewhere else in the
> flow, or the desktop program will suffice. So I'm happy the issue is
> on-hold for now.
>
> I think this is a good example of how the web browser should never have
> evolved into a host for business apps. I think the web browser, HTML, HTTP,
> REST, css and JS have diseased 21st century IT.
>
>
> *Greg K*
>
>> --
> ozdotnet mailing list
> To manage your subscription, access archives: https://codify.mailman3.com/
-- 
ozdotnet mailing list 
To manage your subscription, access archives: https://codify.mailman3.com/ 

Re: Web app large uploads and downloads

[Greg Keogh via ozdotnet]

Folks (welcome to the distant future of 2024)

I'm pretty sure that there is no sensible way to provide a bulk Azure Blob
upload facility in a web hosted app (Wasm, JS or whatever). There could be
thousands of files with a total size up to hundreds of MB. Managers are
currently using a WPF program I created for uploads and it feels like a
perfectly natural process on the desktop, and it's pretty fast using
streams on multi cores (I also optionally check for new or changed so only
*diffs* are uploaded which often saves a lot of time).

None of the components or controls I've seen are designed for huge uploads,
and in any case, I've reported that it's technically and usability
questionable to have "normal" users of the browser app doing this sort of
thing. The boss of the app suite is now considering the bigger picture and
the bulk upload feature may be delayed or moved to somewhere else in the
flow, or the desktop program will suffice. So I'm happy the issue is
on-hold for now.

I think this is a good example of how the web browser should never have
evolved into a host for business apps. I think the web browser, HTML, HTTP,
REST, css and JS have diseased 21st century IT.


*Greg K*

>
-- 
ozdotnet mailing list 
To manage your subscription, access archives: https://codify.mailman3.com/ 

Re: Web app large uploads and downloads

[Tom P via ozdotnet]

I assume you have a client Blazor wasm app that uploads the file to a
server app (ie web api) which would then upload to azure or wherever. Is
this what you are asking?

Thanks
Tom


On Sun, 31 Dec 2023 at 10:19, Craig vN via ozdotnet 
wrote:

> I don't know Blazor, but there are a ton of pure Js file uploaders. One I
> like is https://www.dropzone.dev/ which should be able to be configured
> to upload directly to blog storage, ie not have to send the files to your
> webserver first
> https://stackoverflow.com/questions/35456197/upload-file-directly-to-azure-blob-storage-with-sas-using-dropzone-js
>
> On Sat, Dec 30, 2023 at 8:59 AM Greg Keogh via ozdotnet <
> ozdotnet@ozdotnet.com> wrote:
>
>> Folks (anyone working?)
>>
>> I've been asked to add a feature to a Blazor Webassembly app to allow
>> uploads and downloads of possibly large numbers of files between the local
>> file system and Blob storage. I'm not sure how to implement this feature in
>> a browser hosted app.
>>
>> I wrote a WPF tool for "managers" which does high-performance bulk
>> uploads and downloads with nice progress (the code is trivial on the
>> desktop), but now they want the same feature for "normal" users in the
>> Blazor app. Given how dumb and restricted browser hosted apps are, I don't
>> know how to code this, or if it's even feasible.
>>
>> Are there some tools, techniques or tricks I can apply? Any ideas or
>> suggestions anyone?
>>
>> Thanks,
>> *Greg Keogh*
>>
> --
>> ozdotnet mailing list
>> To manage your subscription, access archives:
>> https://codify.mailman3.com/
>
> --
> ozdotnet mailing list
> To manage your subscription, access archives: https://codify.mailman3.com/
-- 
ozdotnet mailing list 
To manage your subscription, access archives: https://codify.mailman3.com/ 

Re: Web app large uploads and downloads

[Craig vN via ozdotnet]

I don't know Blazor, but there are a ton of pure Js file uploaders. One I
like is https://www.dropzone.dev/ which should be able to be configured to
upload directly to blog storage, ie not have to send the files to your
webserver first
https://stackoverflow.com/questions/35456197/upload-file-directly-to-azure-blob-storage-with-sas-using-dropzone-js

On Sat, Dec 30, 2023 at 8:59 AM Greg Keogh via ozdotnet <
ozdotnet@ozdotnet.com> wrote:

> Folks (anyone working?)
>
> I've been asked to add a feature to a Blazor Webassembly app to allow
> uploads and downloads of possibly large numbers of files between the local
> file system and Blob storage. I'm not sure how to implement this feature in
> a browser hosted app.
>
> I wrote a WPF tool for "managers" which does high-performance bulk uploads
> and downloads with nice progress (the code is trivial on the desktop), but
> now they want the same feature for "normal" users in the Blazor app. Given
> how dumb and restricted browser hosted apps are, I don't know how to code
> this, or if it's even feasible.
>
> Are there some tools, techniques or tricks I can apply? Any ideas or
> suggestions anyone?
>
> Thanks,
> *Greg Keogh*
> --
> ozdotnet mailing list
> To manage your subscription, access archives: https://codify.mailman3.com/
-- 
ozdotnet mailing list 
To manage your subscription, access archives: https://codify.mailman3.com/ 

Re: Web app large uploads and downloads

[kirsten greed via ozdotnet]

How about this one?
https://demos.devexpress.com/blazor/Upload

On Sat, Dec 30, 2023 at 10:06 AM Dr Greg Low via ozdotnet <
ozdotnet@ozdotnet.com> wrote:

> Hi Greg,
>
>
>
> Are you using any Blazor frameworks? Several of them seem to have pretty
> good file upload components. That might give the best experience.
>
>
>
> Regards,
>
>
>
> Greg
>
>
>
> Dr Greg Low
>
>
>
> 1300SQLSQL (1300 775 775) office | +61 419201410 mobile
>
> SQL Down Under | Web: https://sqldownunder.com
> 
>  |
> About Greg:  https://about.me/greg.low
> 
>
>
>
> *From:* Greg Keogh via ozdotnet 
> *Sent:* Saturday, December 30, 2023 8:58 AM
> *To:* ozDotNet 
> *Cc:* Greg Keogh 
> *Subject:* Web app large uploads and downloads
>
>
>
> Folks (anyone working?)
>
>
>
> I've been asked to add a feature to a Blazor Webassembly app to allow
> uploads and downloads of possibly large numbers of files between the local
> file system and Blob storage. I'm not sure how to implement this feature in
> a browser hosted app.
>
>
>
> I wrote a WPF tool for "managers" which does high-performance bulk uploads
> and downloads with nice progress (the code is trivial on the desktop), but
> now they want the same feature for "normal" users in the Blazor app. Given
> how dumb and restricted browser hosted apps are, I don't know how to code
> this, or if it's even feasible.
>
>
>
> Are there some tools, techniques or tricks I can apply? Any ideas or
> suggestions anyone?
>
>
>
> Thanks,
>
> *Greg Keogh*
> --
> ozdotnet mailing list
> To manage your subscription, access archives: https://codify.mailman3.com/
>
-- 
ozdotnet mailing list 
To manage your subscription, access archives: https://codify.mailman3.com/ 

RE: Web app large uploads and downloads

[Dr Greg Low via ozdotnet]

Hi Greg,

Are you using any Blazor frameworks? Several of them seem to have pretty good 
file upload components. That might give the best experience.

Regards,

Greg

Dr Greg Low

1300SQLSQL (1300 775 775) office | +61 419201410 mobile
SQL Down Under | Web: 
https://sqldownunder.com
 | About Greg:  
https://about.me/greg.low

From: Greg Keogh via ozdotnet 
Sent: Saturday, December 30, 2023 8:58 AM
To: ozDotNet 
Cc: Greg Keogh 
Subject: Web app large uploads and downloads

Folks (anyone working?)

I've been asked to add a feature to a Blazor Webassembly app to allow uploads 
and downloads of possibly large numbers of files between the local file system 
and Blob storage. I'm not sure how to implement this feature in a browser 
hosted app.

I wrote a WPF tool for "managers" which does high-performance bulk uploads and 
downloads with nice progress (the code is trivial on the desktop), but now they 
want the same feature for "normal" users in the Blazor app. Given how dumb and 
restricted browser hosted apps are, I don't know how to code this, or if it's 
even feasible.

Are there some tools, techniques or tricks I can apply? Any ideas or 
suggestions anyone?

Thanks,
Greg Keogh
-- 
ozdotnet mailing list 
To manage your subscription, access archives: https://codify.mailman3.com/