Wow, this has some great thoughts. When I wrote up the mail I was thinking about XWiki as basically what it is being used for right now. I want it to be easy for non-devs to build basic applications and easy for the IT departments and project managers (XWiki SAS) to make entire customized services from reusable building blocks.
The most important part in my mind is search. A company can't just throw their crown jewels up on the web and wait for google to index them. We need to be able to take dirty data sets and respond to poorly formed search queries with relevant information quickly. If we can do that then XWiki will be first in the bookmarks list of the work computer, just like google is on everybody's home computer. Second is the user defined model and revision history which make XWiki what it is. With a mutable model, the technical and non-technical users at a customer site can work together applying script and domain knowledge to clean up dirty datasets which might begin their life as a huge word document or excel sheet and then work their way toward becoming a set of XObjects with accompanying data collection APIs and report generation scripts. Without revision history, you don't have the "contribute first ask questions later" culture which makes wiki collaboration work. Third is of course a simple way for project teams to build custom web services. Users have similar needs but they also have edge cases and may want the site to look completely different, this should be easy for developers but need not be within reach for non-developers. I used to think 1 hard drive was plenty for my desktop. When I began to learn about ZFS I realized that actually I had much more data than I had ever thought I had and I really cared about disk failure and silent corruption and the performance of my harddrive was rather poor. Now I know that I need 9 hard drives and 1 ssd cache device. XWiki needs to be the ZFS of industrial intelligence. We need to prove that all companies have much more data than they thought they had, that all employees and customers produce data all the time (the vast majority of which is wasted), and that all data can be converted in to actionable intelligence through collaboration (otherwise it's noise). Once that narrative becomes universal, the need for XWiki will be obvious. More inline: On 03/27/2013 05:30 AM, Ludovic Dubost wrote: > Hi Caleb, > >>From my point of view, before starting to list technologies that would be > great for a next generation XWiki you first need to define what your exact > target is for XWiki. It is very important to do that because when I look at > Jerome's answer I don't necessarly have the feeling he is looking at > exactly the same objective as I would. While his technology choices might > make a lot of sense for an eCommerce platform, they might not in the > Intranet concept. > > This is a first big significant task which depending on the answers will > serve as input on the technology stack to use. Indeed since 2003 when I > started XWiki a lot of new technologies and also use cases or delivery > methods have florished. For instance Cloud is now a very important delivery > method of Web Applications and wether your write for a Cloud service, for > Software deployment or for both this might lead to different technology > choices. Mobile is another key delivery method that was not there in 2003. > On the technology side, SOA, REST, Javascript are everywhere. NoSQL is also > a key technology to consider. > > So let me first try to address the question of what a new XWiki could be > used for, what it is currently used for, what are the strength of XWiki and > it's weaknesses. > > First the initial objective which is the reason for XWiki: > > - building flexible collaborative intranets on top of a Wiki, allowing to > quickly build applications, integrate these applications with the content > > Then a lot of things that users, clients have asked for, developers have > done on their own: > > - providing Knowledge based solutions > - building applications on top of the "Application" development features of > XWiki (the extension of the previous use case) > - providing a full Intranet solution, including what XWiki does natively, > but also providing "Advanced Profiles", "Groups", "Social networks", a set > of Collaborative Applications (Forum, files, etc..) > - building public or internal content web sites with limited editors (CMS > use case) > - building a web application using XWiki as a framework, using many or > little of the XWiki existing functions > - just doing a very nice Wiki in an Enterprise or Public context > - doing any of these in the "Cloud" context (multi tenant) and allowing > users to subscribe to an "use-case" that would be built using the XWiki > technology > > As you can see this is close to the "flavor" discussion we already have and > I've voluntarily listed the first three which are the flavors we are > currently discussing because this is at XWiki SAS what we see organizations > we discuss with are asking for most. > We have seen the other uses cases also but either we had difficulties > addressing them or it was less natural to use XWiki to address them. > > The "framework" use case is a very important one here, because if we think > XWiki's objective is to be a Framework objective for ANY web application, > then this might be incompatible with the Intranet, Knowledge Base and even > the Wiki use case. > For instance Jerome's use case does not fit in the "Intranet" use case and > the problem he has are very different from those you have in an Intranet > environment. > > The "Cloud" one is a very big one also and can lead to radical different > technology choices. As we can see many new cloud services are build around > NoSQL and there are some good reason for that. It's because the Cloud > provider wants a unique platform for all his customers and wants to run the > service in a fully multi-tenant way on top of this unique storage mecanism. > The objective here is to reduce the operating cost to the maximum, in order > to be able to provide a "freemium" service. However this choice might have > a lot of benefits in the "Cloud" context, but it is a radical choice which > has a lot of impact if you also want to provide your service as "Software" > like we do for XWiki. It complexifies the installation, reduces the > features of your storage (very little relational queries), is a radical new > technology to learn to build on which is still very young (choosen a NoSQL > platform today is a very risky bet and supporting multiple technologies is > a nightmare). Now not supporting NoSQL does not necessarly mean you cannot > go "Cloud" but this goes against the way of doing business in the cloud and > complexifies the cloud platform, has scalability issues, etc. Now it's > impossible to ignore "Cloud" but it's important to understand which type of > "Cloud". Is is a unique Cloud or is it SaaS where your objective is to be > able to "host" the service for users and customers. The first choice might > mean going NoSQL, the second might mean sticking with traditional > Databases. The first choice might mean that you will have a hard time > convince "software" users because you also need to convince them to go > NoSQL. I think that when we reach the stage where XWiki is "google for your company". They will suddenly realize they have much more information than mysql could hold without painful replication. Of course for people who believe they will be the next twitter, a massively scalable framework is quite interesting. > > For me the biggest success we have is for "Intranets" and it has always > been the key objective, and what we see is that the "Intranet" use case has > lead us to have to address the "full Intranet" solution but also the "CMS" > one as some organizations want a unique platform for both publishing and > collaboration. There is a lot of pressure also to support all "Social > Network" features as there are obvious bridges between a software that > would be the main places to manage profile, groups and discussions and > another software that want to manage information, knowledge and > applications. > > For me the biggest strength of XWiki is it's flexibility to build > Applications and to Customize it. The first objective of XWiki was to allow > organizations to setup a wiki start sharing knowledge and then extend their > wiki with applications that would organize the information in a better way. > This need is still here today (Podio is a good example of the Application > concept without the Wiki part to glue everything together). I believe many > CMS include more and more "application" features so that you can extend > your web site with more advance organizations and features (but CMS don't > have the same collaborative aspects). The power of XWiki is to allow to do > this fully from your web browser and to do it fast. It is to allow power > users (not developers) to participate because they don't need full > developer skill (this one is very important when it comes to the technology > choices of how XWiki gets extensible). The power of XWiki is to not limit > the customizations capabilities to the content but also allow to customize > the UI by overidding all templates and then morph the initial web site you > created into something else. This power is want allowed XWiki to kind of be > a full "framework" for web development, but it could be that we have pushed > it too far or not far enough. These strength are particularly talking for > some of the users case listed before and for others it can actually be a > weakness. I don't think we will get many technologically savvy customers who want public websites (EG: ecommerce). The market is flooded with competing frameworks and everybody has their own favorite language. Where we win is "the framework with versioning and search built in". > > The weaknesses of XWiki depend on the use cases. For the "Intranet" use > case, our weaknesses have been to not work enough on the UI, to not adapt > enough to the actual intranet use cases (this is where flavors come in). > Another weakness is not enough separation (but not too much) between > content and applications. Applications live in XWiki page but we don't have > enough markers to separate them and not enough "SOA" at the XWiki > Application level. We can fix this. Other weaknesses are how to address > mobile, how to address AJAX development and still allowing power-users (not > developers) to participate in this development. We also took too much time > to build AppWithinMinutes. There are challenges to allowing power-users to > develop and still allow this development to be continued by real developers. > > On the "framework" use-case it's another analysis. Do we need to allow > development in the Wiki if XWiki's job is to compete with web-development > frameworks ? Can we go without hosting the development in an IDE and if so > should it be Eclipse or a Web based IDE ? > Is it compatible to provide development features that are more easily > accessible to non power-user (like velocity is) or can we go with more > advanced technologies. Etc... there are many questions coming from that. I spent some time thinking about this and determined that if we are to support either multi-tenent or multi-node systems, we really need to store any code which might be changed in the database. Even with myxwiki.org, editing a template must be done on both nodes. One option would be for the scripting to be something the user can git clone and push to a jgit server which is integrated in the wiki, similar to how Heroku hosts apps. While that is interesting, it doesn't rank on my list because it is a significant challenge and is of little benefit to the intranet users. Thanks, Caleb > > There are many other very interesting questions that will yield different > answers. For instance should we do a full AJAX UI. What are the > consequences of it. If we go full AJAX, what about URLs ? What about search > engine indexing. Can we do CMS if we are full AJAX. Full AJAX means more > complex to customize also depending on how it's developped. > > To summarize we need to "choose" first what we want XWiki to be. Once we > all agree on this then we can answer how to build the next generation > XWiki. I don't disagree that we should think about this, because it's going > to be 10 years in November since I started building XWiki and of course > technologies evolve and we need to be able to adapt. > > I'm looking forward to your opinions on that. Then I propose we discuss for > each of the use-cases we consider are the future of XWiki what are the > right technologies. Then we also look at the compatibility of technologies > with each other. This will give us a lot of answers. One being in which > direction do we need to put our technological effort on. Another being more > clear about what users and developers can expect from XWiki. > > Ludovic > > > > > > 2013/3/27 Jerome Velociter <[email protected]> > >> Hi Caleb, >> >> Great thread, good exercice. >> >> Like many of us I imagine I've been giving that question some thoughts >> already. I've written an article exactly one year ago about this topic [1] >> (not on rewritting XWiki per-se, but on writing a similar >> application/platform). My thoughts have partly evolved since then, although >> I still think most of what I've laid down there at the time. >> >> Basically I would go like this : >> >> - I would stick to the JVM, and to Java. >> - I would make it a RESTful web service. And probably even several if I >> were to be iso in terms of feature. An example of a feature that I would >> see live in a separated web service (so different process and different API >> host or subdomain) would be anything open office related. It does not >> belong to the base service, and would actually benefit being hosted on the >> same machine as an open office server. This is an example, I believe there >> are other features that do not belong in the base service (Scheduling ? >> Mailing ? etc.). It's SOA, though the base service (the base "wiki engine" >> let say) has to be self-sufficient and offer something that work standalone >> (even if it means no watch list, no office import, etc.) >> - Today I'm not so sure about JAX-RS as I was in my article. It's nice and >> and brings a lot "for free" but now I also think it has some limitations >> that can be penalties, such as limited dynamicity in the way you can play >> with routes. Though maybe this is mitigated by JAX-RS 2.0 filters. >> - Front-end templating would be done with dumb templating, for example >> handlebars.js or dust.js ; with a flexible intermediary layer for preparing >> the "context" needed for such templating mechanisms (written as groovy or >> JS scripts). >> - I would go for a RDBM (like Postgres) without an "industrial ORM". I >> think hibernate and friends bring too much complexity for too little value. >> They tend to impose themselves on the DB design instead of having DB design >> and object design exists independently and meet at the O/R layer. They make >> it hard for the developer to know what is executed against the DB (the "gut >> feeling" can be wrong) and when. Also lazy fetching does not solve any >> actual problem IMHO. >> - I would make anything query related (like searching for objects when >> building applications) happen against the search engine and not the DB. >> - I would use elasticsearch as a search engine. It can be very easily >> embedded (for development and simple deployments) or run as a separate >> cluster, it's schemaless and RESTful by nature. >> - I would embrace JavaScript fully on the "client" for changing state. It >> would be hard for me not to throw AngularJS in the mix. >> - In terms of UX, I would make the "administration" a different UI/UX than >> the wiki itself, and make it an actual "back office". >> - ... it's missing a lot of details, I'll come back if I have some time >> >> Actually I've been trying to put most of those ideas into action for some >> months, into a e-commerce/marketplace platform I'm building [2]. >> >> Now, I'm very eager to read what others think. >> >> Jerome >> >> [1] >> http://velociter.fr/journal/**my-idea-of-a-modern-web-app-**on-the-jvm<http://velociter.fr/journal/my-idea-of-a-modern-web-app-on-the-jvm> >> [2] >> https://github.com/mayocat/**mayocat-shop/<https://github.com/mayocat/mayocat-shop/> >> >> >> >> On 03/27/2013 02:12 AM, Caleb James DeLisle wrote: >> >>> This is the premise, you are going to write a new XWiki. >>> You are not bound by any backward compatibility requirements. >>> Use any technology or combination of technologies. >>> PHP? C++? Golang? Node.js? Java? Your call! >>> >>> Describe it in as much detail as possible. >>> What frameworks will you use? What ORM? what databases? >>> >>> Why will it perform well? >>> How will you get new features into the hands of users quickly? >>> How will you avoid this: >>> http://steve-yegge.blogspot.**ca/2007/12/codes-worst-enemy.**html<http://steve-yegge.blogspot.ca/2007/12/codes-worst-enemy.html> >>> and this: http://www.laputan.org/mud/ >>> >>> Think big and go wild! >>> >>> Thanks, >>> Caleb >>> >>> ______________________________**_________________ >>> devs mailing list >>> [email protected] >>> http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs> >>> >> >> ______________________________**_________________ >> devs mailing list >> [email protected] >> http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs> >> > > > _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
