update : it turns out Horde groupware supports the free-to-use-commercially PostGreSQL, and that does scale at least a little bit, with writes (containing data from for instance any of your IMAP servers through Horde Groupware to a PostGreSQL server) going to 1 master postgresql server, which then writes to multiple different machines, which are then used for the read commands. And all of the stack that Horde needs, an LDAP server, an full SMTP + IMAP stack that uses the LDAP server for authentication, can also run on a single core-i5 machine with say 16GB of RAM, running ubuntu.com's free OS. In addition to running nginx and/or Apache plus PHP and the rest needed for my web-app.
Whether this works well in the real world, well, it will work. how well it'll work, is hard to predict, and valuable information. so what i'm trying to say, briefly this time, is that there is still a business opportunity to be had with this class of functionality, and *if* you choose to base your stuff on Horde Groupmail, you can save yourself a ton of time. All that you'd need to write are the installation scripts, server-adding scripts, a cloud strategy, identification of bottlenecks (which should be measured via scripting to the operating system, stored, and displayed, even emailed to site owners), and plans and scripts to easily add servers, *even* by people with modest techincal skills, so a cheap cloud system based on ISPs and fiber-ISPs, rather than (relying solely on) expensive dedicated hosting centers or cloud services that increase their costs as your clientele becomes more active, can evolve not just for this functionality, but basically serve as an example for other functionality like blog-content, forum-content, photo and video hosting too. For any functionality, the installation scripts and server-adding scripts are the most important to focus on, followed by the performance test scripts. The actual GUI of the functionality should be your lowest priority. Horde is theme-able, by the way. So if you want to jump into providing this functionality, i'd now recommend you write all these mentioned scripts in say python or (better, because it can be accessed over the web more easily) PHP, and focus initially on leveraging Horde for the actual functionality, before even trying to outdo the Horde team when it comes to user-interface gimmicks. Keep in mind, the Horde project is actively developed with a version 6 underway. On Thu, Nov 1, 2018 at 2:11 PM Rene Veerman <seductivea...@gmail.com> wrote: > i've thought about it some more, have looked at the demo of Martin's > ember, and i'm currently leaning in the direction of something like Horde > groupware (which is LGPL), which supports among many other things, full > IMAP email. > and i've found (from the 90s, so no doubt better organized these days) a > solution to scale up IMAP using an LDAP server, see > https://www.horde.org/papers/Scalable_webmail_HOWTO.html > > however. Horde uses sql, and i know that mysql cluster version costs 10 > grand per year(!), so that's a no-no for me. > i aim to keep costs, especially recurring costs, as low as possible for my > pretty CMS seductiveapps.com. > > and i feel that once again i'm ahead of the curve when it comes to my > functionality and performance desires, and the state of the open source > world. > > i'd like to announce the real opportunity for anyone who loves couchdb, to > build a non-GPL, LGPL/MIT/Apache-licensed, truly free, groupware solution > (just expose it via HTTP somehow, so that it can be loaded in an iframe > hosted on the same domain as whatever it's embedded into) that uses the > above scalable LDAP + IMAP setup to facilitate email to the outside world. > you could turn this into a cloud-business much like tiny.cloud does for the > free and very useful and cool tinymce rich texteditor which i use in my > CMS, saving a guy like me the hassle and especially the learning and > testing curves of hosting the groupware part of my CMS myself. > personally, i'm probably going to research all of this myself, but since > my CMS is my only hope to get out of relative poverty, i'm going to publish > whatever i write under my own http://seductiveapps.com/LICENSE.txt to > fetch 1% of revenue earned *in commercial operations*, (or less for really > large revenue creators) and which sticks to terminology very similar to the > MIT license for all other legal matters. In other words, you can use my > stuff, add value to it in the form of code or artwork, and add your own > license terms to any client you want to sell your seductiveapps based > product to. > > i say this, not to generate clients for myself. i say this, because ever > since "the cathedral and the bazaar" was published (as a book, which i read > and have), which deals with how to actually make money with open source, > few have shown publicly (to my knowledge), how to do this. > > what i write above here, is part of the solution i think. stick to > opensource that may be used commercially and which may be modified without > legal problems, and keep your costs for running your (web-)software, > especially recurring costs, as low as possible at all times, while making > sure your software *can scale* to many machines spread intelligently out > over the world (for more info, see my README files at > https://gitlab.com/seductiveapps/seductiveapps/blob/master/README-001-setup.txt > and > https://gitlab.com/seductiveapps/seductiveapps/tree/master/__README__INSTALLATION_INSTRUCTIONS > ), with the easiest and best-supported or best-proven stack of opensource > software, to keep the learning curve and operating complexity of your stuff > as low as possible. > > you do all that, then here's your real chance to fill a big hole in the > market : groupware, starting with email that scales (probably close to how > that article from 1999 listed above here specifies, and ... basically a > bunch of installer scripts for ubuntu would do that trick nicely these > days, it *can* all be run on one server (important for startups growing a > clientele), and you wouldn't have to support other operating systems unless > you easily know how to and want to), > followed by live chat between users with the option of having an email log > of any chat sent to an IMAP server, > and moving on to things like agenda features that can sync to any > smartphone. > > this is a moving market, you wouldn't have to write everything yourself > (for your treeview needs, look no further than jstree, and for your rich > text editing needs the latest tinymce is great even on smartphones), > and for people with more money than time or ability to find technical > talents, you could make decent money providing all of this as a package > that can be loaded up in an iframe specified Access-Control-Origin to that > site that is your client. > > the advantages of going LGPL or MIT or Apache license with *new groupware > that solves the very expensive SQL server clustering (scalability) problem > in Horde groupware* (the only commerically-free-to-use opensource > groupware that i could find yesterday by the way)? > you'd gain more clients and if you are smart and build simple to read code > that uses plugins for specific functions, then you'd have the biggest > chance for others to write you free bugfixes and feature enhancements > created and donated back to you and your project via your license. > i'd recommend strongly you host yourself on github or gitlab and enable > the "issues" tab for your project, as well as provide an email list (which > would be useful as a feature of the groupware you're building, but which > can be based on standard and well-tested, well-supported, ubuntu software > packages). > > lastly, you need to write test-scripts that solve the question : how much > computing power do i have to buy, what is that going to cost me, and the > same for bandwidth (a simple ADSL contract of 50 bucks a month supports all > of my home browsing *and* my server hosting costs, try beating that; i'd > love to hear how you did it :).... to support how many clients that are > this or that much active per percentage of simulated users. > i know. that's a real bitch to write, a real distraction from developing > features, but i'm going to have to do the same for my CMS. You can't build > features these days without the ability to prove how many users, active > users, and very active users, you can support with how much hardware, > organized specifically in this or that way. > > you have to explain and document all these variables not just in your > documentation, but in test-scripts as well. > test-scripts that at least note the CPU type, CPU GHz count, CPU core > count, operating system version, all version numbers of your other critical > software (like couchdb), and of course how much RAM the machine has in > total, and how much free RAM at start of the test. > > you can then publish these test-results on your homepage, and by having > specific test-scripts (that actually send back their results to your server > automatically), others can test on their hardware how many daily users they > would support. > you have to specify, per any of multiple time-ranges, total time-range > being flexible as well, the : total number of users, the number of normal > users and all the counts of operations such a normal user would make in the > specified time-range (executed as a web-browser would execute it, but > always starting with a high-level operation like requesting all the emails > in a certain inbox for a certain user. yes, you'll need to build tests for > each feature that is often used, at least) (and these numbers can be > measured as well in the real world, make sure you keep logs of that > somewhere where they can be found by the site operator too), and the same > for 'active users' and 'very active users'. you'll probably need to specify > a random timelength (from a specified time range again) between requests > made by these users, and it wouldnt hurt to make your performance tests in > a way a user would use the site. for email, that is : how long to request > the list of emails in an IMAP inbox of a certain size (and whether you pull > all IMAP data into couchdb is up to you : IMAP + LDAP does scale already), > then how long to store and send an email (to a test IMAP bucket of course) > with or without smartphone pictures, then how long to "read" (request the > contents of) several emails in such an inbox. > > this may seem like a lot of work and hard work. but actually it's just > accountancy level math : add, subtract, multiply, divide. that's it. > > it's a real chance for your own successful opensource + cloud company, and > i'll offer anyone partially interested the deal that if if their stuff > meets all of these requirements i list above here, you can use my > seductiveapps.com for free on your site, forever, if you like. > but the GUI part of your HTTP iframe-able application, must include theme > abilities. > once you're all done, and then after a long rest get bored, you can always > add a theme editor, like i've done and am doing for my CMS. > this way you can enable non-technical artists to create some free > cool-looking themes for you, which will add much value to your groupware > product, and provide the many who can't afford to pay big bucks for > software, to do something in return for you. > i'd be delighted to share back a good-looking semi-transparent theme for > your groupware, and all your users/clients.. > > over the next few days, i'll decide how far to take this. i can do this, > but i'd have to stick to my custom license that's not entirely free for > commercial use (1% of your revenue made on a site that uses my CMS please), > and work on it for about a year, maybe 2 years even, > or i can focus on the rest of my CMS' functionality, and just wait for > someone or some group to fill this hole in the market. so i'm sending this > email to the 2 groups i know to hold people who might be up for this > callenge : the couchdb user, and php-general, mailinglists. > once again, MySQL cluster version costs frigging 10 grand USD per year! > that would kill my entire project and company. so would using Horde > Groupware with just one MySQL server. and i didn't spend years building my > stuff just to step into a comfy pitfall like Horde is today. > > and i'm far from the only CMS builder out there. there's a real market for > this solution. > if you'd keep your solution easy to install (and all it's components too > ofcourse), then guys like me *will* be interested in your commercial > cloud-hosting offerings (provided they dont scale up in cost as usage > increases, like Amazon S3 does), to save ourselves headaches and growing > pains as we deal with increasing numbers of real customers. > > feel free to email me to pick my brain some more, if that's what you feel > you need. > or just reply to this list. > > > > On Fri, Oct 26, 2018 at 10:34 AM Martin Broerse <martin.broe...@gmail.com> > wrote: > >> Hi Rene, >> >> Perhaps take a look at https://github.com/broerse/ember-cli-blog and >> change >> your plan to make it offline-first. >> >> - Martin >> >> >> On Fri, Oct 26, 2018 at 7:32 AM Rene Veerman <seductivea...@gmail.com> >> wrote: >> >> > my current, newbie strategy, is to have multiple databases per user each >> > dedicated to a single feature, like customizations of the look of a >> page, >> > or the blog entries. >> > this has the advantage of easy permissions enforcement, >> > but the disadvantage of a worse than exponential explosion of the >> number of >> > databases as features requiring a new database per user/role, and the >> > number of users, grows. >> > >> > and quite frankly i'm a bit afraid to just start designing something on >> my >> > own, knowing next to nothing about the nitty gritty of replication in >> the >> > real world. >> > >> > or should i just not be worried about 20 million databases per 1 million >> > users? there is that b-tree feature which i assume is used for >> table-name >> > to table data access too? >> > >> > On Fri, Oct 26, 2018 at 7:19 AM Rene Veerman <seductivea...@gmail.com> >> > wrote: >> > >> > > Now that i've got blogging powered by couchdb and some other stuff >> > powered >> > > by couchdb done for my seductiveapps.com platform, >> > > >> > > i'm looking to start on email and chat features, between couchdb users >> > and >> > > even between roles or between a user and someone from a role. >> > > >> > > a role like 'law enforcement', because new anti-terror laws here >> require >> > > that i build in a backdoor to any system that exposes blogging >> features >> > to >> > > the world. >> > > >> > > plus, i need reliable scalable email and chat features anyways for the >> > > more advanced future features of my cms (webshop, forum, etc). >> > > >> > > so, i'm looking for smart db design ideas (i use php-couchdb and >> > pouchdb), >> > > and if any library to do this exists that's free to use commercially, >> i'd >> > > love to hear about that too. >> > > >> > >> >
