Re: S3 Object Storage
Hello Koby, On 2020-04-06 07:07, Koby Peleg Hen wrote: Hello all , I am currently on this , Please Janos , make your decision. Since I have looking at your project for very long time , And my client begin to ask for. It is very simple to implenent , because there is simple convertor from regular file system to S3. If you decided not to , please let us all know. the answer is no for now. I've googled the topic, and found s3fs (perhaps you referred to the same project). Found a few articles that might come handy: https://cloud.netapp.com/blog/amazon-s3-as-a-file-system https://medium.com/swlh/using-s3-just-like-a-local-file-system-in-python-497737783f11 I encourage you and other piler users as well to test it. Perhaps it's a good enough solution, and works with piler properly. If so, then I'll update the docs with your findings. In this case I will need to implement it by my self by the near future. I have a lot of respect to your project and your time and the willing to get pay for it. But if some one whould like to pay , is for the support not for feature. Please take a look on the proxmox project. I just did. And looked at nginx plus as well. Some projects monetize support, others the features as well. By my self , and my clients , I do not mean to take off without S3. I'd say give a shot for s3fs, and see how it works. I'm not against it (or any usable solution) by any means. Janos SUTO
Re: S3 Object Storage
Hi Janos, My preferred provider is Linode, I've had a couple VM's with them for years and they've always been top-notch. I won't mention the other provider (I'm not one to bash) currently hosting my Piler instance that I see service drops on almost every morning, but I will say that (failing to have S3 support) I'd be looking to migrate from them to RamNode as they appear to have decent reviews and a "spinning disk" option with a good amount of space (comparable to that of the current host). So if you/anyone has any praise or warnings of RamNode hosting Piler, I'm all ears. I have not yet had to dabble in the C code for anything, but am familiar with C/C++ and a few other various languages, so I always like to have the option. It's good to know the web UI is still open in the enterprise version. As for trying enterprise at this time, I suppose I'll try the RamNode route first with spinning disk block storage. If the provider isn't as stable as I'd like I may have to reconsider, but for now it will be a matter of cost of licensing Piler purely for the S3 capability versus minimizing my hosting provider options to those with larger storage options. If I require a more stable host and can't find any with inexpensive storage, it just comes out to a cost calculation between block storage cost in a provider versus Piler enterprise costs for S3 compatibility... unless of course you come out with a discounted S3 "add-on license" for just that feature that makes the math a no-brainer. I've been using Piler for so many years personally, I should probably just start pushing clients toward it, but I've had some issues with it missing mails due to service outages, failing to load the UI unless the service is restarted (after running for several days), failing to clear the sph_index queue on a semi-regular basis, and a few other quirks along the way that made me pause before offering it as an actual service in a customer-facing manner. I've automated most of that fixing itself with Zabbix monitoring, but some issues remain. Honestly though, I've attributed most if not all of that to the current VPS host. If the new version I install on a new host sees all those issues go away, and I'm comfortable having potential clients log into the interface and know something random isn't going to happen to them, the easy call is to push clients toward it and get an enterprise license anyway. So here's hoping that's the case! I'd love to get rid of the proprietary options I've got some clients on and move everyone to Piler if possible. Thanks again Janos! On Mon, Apr 6, 2020 at 3:11 AM wrote: > > Hello Ryan, > > On 2020-04-06 00:45, Ryan Blenis wrote: > > > > Thank you as always for your quick and in-depth response. I certainly > > understand the delicate balance between FOSS and enterprise paywalls > > and feature-sets, and I thank you for even having an open source > > option to begin with! I'm just planning a migration to a new server > > (for when my current OS is EOL) and wanted to move the hosting to one > > with better support that doesn't over-provision the hosts resources as > > I've seen on the current host, but HDD space isn't anywhere near as > > inexpensive as it is at the current company. I was just thinking I > > could have the best of both worlds with S3 storage integrated. Many of > > the FOSS/enterprise paywalls in other projects are based on LDAP > > integration, multi-tenancy, pre-packages binaries, support levels, > > etc. that many business need, whereas object storage vs block storage > > felt like more of a separate paywall in terms of cloud providers, so I > > just figured I'd ask if it was coming to the FOSS version. Also, I > > it's ok to ask, and I always appreciate the feedback from piler users. > Btw. can you share the names of these 2 providers? I'm curious about > their pricing. And perhaps the piler community may suggest some better > alternatives both in terms of hw performance and storage pricing. > > > > like to be able to modify the code if a customization can be made, > > which I assume would not be the case with enterprise, correct? > > Not sure if you fix the C source code before compiling. If not, then > it's probably not a big deal that you can't modify the enterprise > edition > binaries. However, the gui part is still the usual php based stuff, > it's not obfuscated, so you can fix it if you really want to do that. > It's your archive after all. Custom css is supported out of the box. > > > Also, if I try the enterprise on the new host, is there a path to > > downgrade to the FOSS version? Or would starting back from scratch be > > the only option? > > Unfortunately no. The enterprise edition handles attachments in a > different > way: it doesn't need the attachment table to retrieve the message with > its > attachments. However, it's incompatible with the open source version. > > So if you decide later that it's not what you want then you should > export > all emails, install the
Re: S3 Object Storage
Hello Ryan, On 2020-04-06 00:45, Ryan Blenis wrote: Thank you as always for your quick and in-depth response. I certainly understand the delicate balance between FOSS and enterprise paywalls and feature-sets, and I thank you for even having an open source option to begin with! I'm just planning a migration to a new server (for when my current OS is EOL) and wanted to move the hosting to one with better support that doesn't over-provision the hosts resources as I've seen on the current host, but HDD space isn't anywhere near as inexpensive as it is at the current company. I was just thinking I could have the best of both worlds with S3 storage integrated. Many of the FOSS/enterprise paywalls in other projects are based on LDAP integration, multi-tenancy, pre-packages binaries, support levels, etc. that many business need, whereas object storage vs block storage felt like more of a separate paywall in terms of cloud providers, so I just figured I'd ask if it was coming to the FOSS version. Also, I it's ok to ask, and I always appreciate the feedback from piler users. Btw. can you share the names of these 2 providers? I'm curious about their pricing. And perhaps the piler community may suggest some better alternatives both in terms of hw performance and storage pricing. like to be able to modify the code if a customization can be made, which I assume would not be the case with enterprise, correct? Not sure if you fix the C source code before compiling. If not, then it's probably not a big deal that you can't modify the enterprise edition binaries. However, the gui part is still the usual php based stuff, it's not obfuscated, so you can fix it if you really want to do that. It's your archive after all. Custom css is supported out of the box. Also, if I try the enterprise on the new host, is there a path to downgrade to the FOSS version? Or would starting back from scratch be the only option? Unfortunately no. The enterprise edition handles attachments in a different way: it doesn't need the attachment table to retrieve the message with its attachments. However, it's incompatible with the open source version. So if you decide later that it's not what you want then you should export all emails, install the open source edition, and import those emails. Though, I'm confident that you'll like it, it's not inferior to the open source edition. Janos SUTO On Sun, Apr 5, 2020 at 7:16 AM wrote: Hello Ryan, On 2020-04-05 10:55, Ryan Blenis wrote: I see the enterprise version you offer it says one of the features piler has is S3 object storage, which looks like it was originally requested on the mailing list in 2018: https://www.mail-archive.com/piler-user@list.acts.hu/msg01335.html Does this mean that it is fully implemented now? And if so, will we yes, it's fully implemented by now. ever see this in the open source version? Given the lower prices on S3 compatible storage versus block storage and the unchanging nature of the files, this would be great to have as an option that can reduce hosting costs from what is now being used for [comparatively] expensive block storage. well, I've been thinking about it for a while, and still not decided, it's a difficult business decision, so let me share my current position on the matter. TL;DR: not in the near future. I'm really proud that the open source edition has gained a pretty nice traction, and I decided to somehow monetize its popularity. So I've forked piler and built an enterprise edition along with some commercial services. To make the enterprise edition successful, I must offer some features that are worth to pay for that is not available for free in the open source edition. I believe that the S3 storage support you asked for is such an attractive feature, so I'll keep it for the enterprise edition only, not sure how long. However, I'm well aware of that the S3 costs are significantly lower than block storage costs at cloud hosting companies. I encourage you to try the enterprise edition, and see if it works for you. If so, and you like it, then perhaps we could figure out something viable to make the transition from open source to enterprise. I'd like to emphasize that I don't want to force you in this direction, and it's totally ok to decline. Let me know what you (and other piler users as well) think about the matter. Janos
Re: S3 Object Storage
Hello all , I am currently on this , Please Janos , make your decision. Since I have looking at your project for very long time , And my client begin to ask for. It is very simple to implenent , because there is simple convertor from regular file system to S3. If you decided not to , please let us all know. In this case I will need to implement it by my self by the near future. I have a lot of respect to your project and your time and the willing to get pay for it. But if some one whould like to pay , is for the support not for feature. Please take a look on the proxmox project. By my self , and my clients , I do not mean to take off without S3. Please let us all know what have you decide. Thak you all for your support and your time. הורד את TypeApp עבור Android ב-6 באפר׳ 2020, 1:47, ב-Ryan Blenis , 1:47 כתב: >Hi Janos, > >Thank you as always for your quick and in-depth response. I certainly >understand the delicate balance between FOSS and enterprise paywalls >and >feature-sets, and I thank you for even having an open source option to >begin with! I'm just planning a migration to a new server (for when my >current OS is EOL) and wanted to move the hosting to one with better >support that doesn't over-provision the hosts resources as I've seen on >the >current host, but HDD space isn't anywhere near as inexpensive as it is >at >the current company. I was just thinking I could have the best of both >worlds with S3 storage integrated. Many of the FOSS/enterprise paywalls >in >other projects are based on LDAP integration, multi-tenancy, >pre-packages >binaries, support levels, etc. that many business need, whereas object >storage vs block storage felt like more of a separate paywall in terms >of >cloud providers, so I just figured I'd ask if it was coming to the FOSS >version. Also, I like to be able to modify the code if a customization >can >be made, which I assume would not be the case with enterprise, correct? > >Also, if I try the enterprise on the new host, is there a path to >downgrade >to the FOSS version? Or would starting back from scratch be the only >option? > >Thank you as always again. > >On Sun, Apr 5, 2020 at 7:16 AM wrote: > >> >> Hello Ryan, >> >> On 2020-04-05 10:55, Ryan Blenis wrote: >> > >> > I see the enterprise version you offer it says one of the features >> > piler has is S3 object storage, which looks like it was originally >> > requested on the mailing list in 2018: >> > https://www.mail-archive.com/piler-user@list.acts.hu/msg01335.html >> > >> > Does this mean that it is fully implemented now? And if so, will we >> >> yes, it's fully implemented by now. >> >> >> > ever see this in the open source version? Given the lower prices on >S3 >> > compatible storage versus block storage and the unchanging nature >of >> > the files, this would be great to have as an option that can reduce >> > hosting costs from what is now being used for [comparatively] >> > expensive block storage. >> >> >> well, I've been thinking about it for a while, and still not decided, >> it's a difficult business decision, so let me share my current >position >> on the matter. >> >> TL;DR: not in the near future. >> >> I'm really proud that the open source edition has gained a pretty >nice >> traction, and I decided to somehow monetize its popularity. >> >> So I've forked piler and built an enterprise edition along with some >> commercial >> services. To make the enterprise edition successful, I must offer >some >> features >> that are worth to pay for that is not available for free in the open >> source edition. >> >> I believe that the S3 storage support you asked for is such an >> attractive feature, >> so I'll keep it for the enterprise edition only, not sure how long. >> >> However, I'm well aware of that the S3 costs are significantly lower >> than block >> storage costs at cloud hosting companies. I encourage you to try the >> enterprise >> edition, and see if it works for you. If so, and you like it, then >> perhaps we >> could figure out something viable to make the transition from open >> source to >> enterprise. I'd like to emphasize that I don't want to force you in >this >> direction, >> and it's totally ok to decline. >> >> Let me know what you (and other piler users as well) think about the >> matter. >> >> Janos >> -- test text sig
Re: S3 Object Storage
Hi Janos, Thank you as always for your quick and in-depth response. I certainly understand the delicate balance between FOSS and enterprise paywalls and feature-sets, and I thank you for even having an open source option to begin with! I'm just planning a migration to a new server (for when my current OS is EOL) and wanted to move the hosting to one with better support that doesn't over-provision the hosts resources as I've seen on the current host, but HDD space isn't anywhere near as inexpensive as it is at the current company. I was just thinking I could have the best of both worlds with S3 storage integrated. Many of the FOSS/enterprise paywalls in other projects are based on LDAP integration, multi-tenancy, pre-packages binaries, support levels, etc. that many business need, whereas object storage vs block storage felt like more of a separate paywall in terms of cloud providers, so I just figured I'd ask if it was coming to the FOSS version. Also, I like to be able to modify the code if a customization can be made, which I assume would not be the case with enterprise, correct? Also, if I try the enterprise on the new host, is there a path to downgrade to the FOSS version? Or would starting back from scratch be the only option? Thank you as always again. On Sun, Apr 5, 2020 at 7:16 AM wrote: > > Hello Ryan, > > On 2020-04-05 10:55, Ryan Blenis wrote: > > > > I see the enterprise version you offer it says one of the features > > piler has is S3 object storage, which looks like it was originally > > requested on the mailing list in 2018: > > https://www.mail-archive.com/piler-user@list.acts.hu/msg01335.html > > > > Does this mean that it is fully implemented now? And if so, will we > > yes, it's fully implemented by now. > > > > ever see this in the open source version? Given the lower prices on S3 > > compatible storage versus block storage and the unchanging nature of > > the files, this would be great to have as an option that can reduce > > hosting costs from what is now being used for [comparatively] > > expensive block storage. > > > well, I've been thinking about it for a while, and still not decided, > it's a difficult business decision, so let me share my current position > on the matter. > > TL;DR: not in the near future. > > I'm really proud that the open source edition has gained a pretty nice > traction, and I decided to somehow monetize its popularity. > > So I've forked piler and built an enterprise edition along with some > commercial > services. To make the enterprise edition successful, I must offer some > features > that are worth to pay for that is not available for free in the open > source edition. > > I believe that the S3 storage support you asked for is such an > attractive feature, > so I'll keep it for the enterprise edition only, not sure how long. > > However, I'm well aware of that the S3 costs are significantly lower > than block > storage costs at cloud hosting companies. I encourage you to try the > enterprise > edition, and see if it works for you. If so, and you like it, then > perhaps we > could figure out something viable to make the transition from open > source to > enterprise. I'd like to emphasize that I don't want to force you in this > direction, > and it's totally ok to decline. > > Let me know what you (and other piler users as well) think about the > matter. > > Janos >
Re: S3 Object Storage
Hi, I think it is important to have an enterprise solution and I would say S3 is a great feature to begin with. We need to keep Piler alive. Open Source projects do not live from warm words or "it is good for your reputation when we use your software". :) Piler is probably the best open source archiver. Let us not lose it. Thank you, Janos! Mit besten Grüßen André Peters > Am 05.04.2020 um 13:16 schrieb "s...@acts.hu" : > > > > Hello Ryan, > >> On 2020-04-05 10:55, Ryan Blenis wrote: >> I see the enterprise version you offer it says one of the features >> piler has is S3 object storage, which looks like it was originally >> requested on the mailing list in 2018: >> https://www.mail-archive.com/piler-user@list.acts.hu/msg01335.html >> Does this mean that it is fully implemented now? And if so, will we > > yes, it's fully implemented by now. > > >> ever see this in the open source version? Given the lower prices on S3 >> compatible storage versus block storage and the unchanging nature of >> the files, this would be great to have as an option that can reduce >> hosting costs from what is now being used for [comparatively] >> expensive block storage. > > > well, I've been thinking about it for a while, and still not decided, > it's a difficult business decision, so let me share my current position > on the matter. > > TL;DR: not in the near future. > > I'm really proud that the open source edition has gained a pretty nice > traction, and I decided to somehow monetize its popularity. > > So I've forked piler and built an enterprise edition along with some > commercial > services. To make the enterprise edition successful, I must offer some > features > that are worth to pay for that is not available for free in the open source > edition. > > I believe that the S3 storage support you asked for is such an attractive > feature, > so I'll keep it for the enterprise edition only, not sure how long. > > However, I'm well aware of that the S3 costs are significantly lower than > block > storage costs at cloud hosting companies. I encourage you to try the > enterprise > edition, and see if it works for you. If so, and you like it, then perhaps we > could figure out something viable to make the transition from open source to > enterprise. I'd like to emphasize that I don't want to force you in this > direction, > and it's totally ok to decline. > > Let me know what you (and other piler users as well) think about the matter. > > Janos >
Re: S3 Object Storage
Hello Ryan, On 2020-04-05 10:55, Ryan Blenis wrote: I see the enterprise version you offer it says one of the features piler has is S3 object storage, which looks like it was originally requested on the mailing list in 2018: https://www.mail-archive.com/piler-user@list.acts.hu/msg01335.html Does this mean that it is fully implemented now? And if so, will we yes, it's fully implemented by now. ever see this in the open source version? Given the lower prices on S3 compatible storage versus block storage and the unchanging nature of the files, this would be great to have as an option that can reduce hosting costs from what is now being used for [comparatively] expensive block storage. well, I've been thinking about it for a while, and still not decided, it's a difficult business decision, so let me share my current position on the matter. TL;DR: not in the near future. I'm really proud that the open source edition has gained a pretty nice traction, and I decided to somehow monetize its popularity. So I've forked piler and built an enterprise edition along with some commercial services. To make the enterprise edition successful, I must offer some features that are worth to pay for that is not available for free in the open source edition. I believe that the S3 storage support you asked for is such an attractive feature, so I'll keep it for the enterprise edition only, not sure how long. However, I'm well aware of that the S3 costs are significantly lower than block storage costs at cloud hosting companies. I encourage you to try the enterprise edition, and see if it works for you. If so, and you like it, then perhaps we could figure out something viable to make the transition from open source to enterprise. I'd like to emphasize that I don't want to force you in this direction, and it's totally ok to decline. Let me know what you (and other piler users as well) think about the matter. Janos
