Hey Shubham, I have tried initialising with 1 pod and scaling it up to multiple pods, also tried session affinity, and some other cases but none of them worked.
Thanks, Ashish On Fri, 19 Jul 2019 at 4:51 PM, Shubham Jain <jainshubha...@gmail.com> wrote: > Hey Ashish, > > I don’t understand XWiki well enough to fully understand the implications > of running a clustered setup by sharing file system. > > Can you try keeping the deployment size initially 1, and then later try to > increase the size? Does that also cause problems? > > Best, > Shubham. > > > On 19-Jul-2019, at 4:03 PM, Ashish Sharma <sharma.ashish...@gmail.com> > wrote: > > > > Hey Shubham, > > > > I have moved the solr folder as exposed by the docker container to the > file > > system. I think in-memory cache is creating problem. > > > > I started 3 container of xwiki with shared file storage, but all three > were > > trying to initiate the postgress database, that is causing problem. > > > > I want to know few things about xwiki:- > > > > 1. Do we use java threads ? -> if we do then filesystem locking should > have > > been implemented for that? > > > > 2. Other stateful things in our application like caching or other > > filestorage that should be externalise? > > > > > > Thanks, > > Ashish > > > > On Fri, 19 Jul 2019 at 3:59 AM, Shubham Jain <jainshubha...@gmail.com> > > wrote: > > > >> Hey Ashish, > >> > >> I think for now you could stick to deployment. Have you been ever to > >> discover the data you need to move to the shared file system? Is the > >> in-memory caching mechanism causing problems? > >> > >> @Thomas/Vincent - Can you please help us understand the implications of > >> running XWiki in a clustered mode by sharing data volumes? > >> > >> Best, > >> Shubham. > >> > >> > >> Best, > >> Shubham? > >> > >>> On 19-Jul-2019, at 5:35 AM, Ashish Sharma <sharma.ashish...@gmail.com> > >> wrote: > >>> > >>> Resent the email at the mail thread. > >>> > >>> On Fri, Jul 19, 2019 at 2:01 AM Ashish Sharma < > >> sharma.ashish...@gmail.com> > >>> wrote: > >>> > >>>> Hello everyone, > >>>> > >>>> I am trying to implement xwiki helm chart in High availability but > >> facing > >>>> some problems. > >>>> > >>>> How we can implement High Availability(HA) in Kubernetes:- > >>>> > >>>> 1. Through Deployment:- In this approach we would use a shared file > >>>> storage, that would get attached to multiple pods(or containers) > >>>> 2. Through StatefulSet:- In this approach every pod(or container) > would > >>>> have it’s own different volume and state, which should be synchronised > >> by > >>>> an consensus algorithm > >>>> > >>>> Problem faced in xwiki case:- > >>>> > >>>> 1. In deployment case we need a locking mechanism for our filesystem, > >>>> moreover caching and states should also be maintained outside the > >> container. > >>>> 2. In StatefulSet the consensus algorithm is not present so it is > >>>> currently not possible. > >>>> > >>>> Could anyone help me through it. > >>>> > >>>> Thanks, > >>>> Ashish > >>>> > >>>> On Wed, 10 Jul 2019 at 10:32 AM, Shubham Jain < > jainshubha...@gmail.com> > >>>> wrote: > >>>> > >>>>> Hey Ashish, > >>>>> > >>>>> I think Vincent/Thomas can help with the PR on the xwiki-docker > >> project. > >>>>> > >>>>> Regarding adding docs for High-Availability setup using shared data > >>>>> volumes, I think you can use could create a docs folder in the GitHub > >> repo, > >>>>> and add the docs in Markdown format. I think you can also document it > >> in > >>>>> XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart > >>>>> should support enabling HA, and then passing a storage class which > >> supports > >>>>> shareable volumes to Pods, and then you can document examples for > using > >>>>> this with something like Rook+GKE setup. > >>>>> > >>>>> Best, > >>>>> Shubham. > >>>>> > >>>>> > >>>>>> On 08-Jul-2019, at 1:57 PM, Ashish Sharma < > sharma.ashish...@gmail.com > >>> > >>>>> wrote: > >>>>>> > >>>>>> Hello everyone, > >>>>>> > >>>>>> I require the xwiki solr docker container to do progress with my > >> work. I > >>>>>> have made a pull request > >>>>>> <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the > >> same. > >>>>>> Please look into it. > >>>>>> > >>>>>> Moreover, I was working with Rook(Ceph Operator) for a shared file > >>>>> system > >>>>>> which would be a better approach to deploy our app on production. > So I > >>>>>> think we should document it somewhere. Can anyone help me where > should > >>>>> I do > >>>>>> that. It may contain the details to configure rook-ceph, and then > >>>>>> connecting it. > >>>>>> > >>>>>> P.S. I am writing a blog at Medium on how to configure rook on > Google > >>>>>> Kubernetes Engine. We could also use that? > >>>>>> > >>>>>> Thanks, > >>>>>> Ashish > >>>>>> > >>>>>> On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma < > >>>>> sharma.ashish...@gmail.com> > >>>>>> wrote: > >>>>>> > >>>>>>> Hi Shubham, > >>>>>>> > >>>>>>> Thank you Shubham, I will start working on adding test cases. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Ashish > >>>>>>> > >>>>>>> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain < > >> jainshubha...@gmail.com> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Hey Ashish, > >>>>>>>> > >>>>>>>> Thanks for the update. > >>>>>>>> > >>>>>>>> Adding our chart to incubator section of helm community charts > >>>>> repository > >>>>>>>> sounds great. Although (unlike Dockerhub) this repo is for > community > >>>>> charts > >>>>>>>> and not really official charts of the respective installations, > but > >>>>>>>> supporting this chart in the community chart repo should help with > >> the > >>>>>>>> visibility. > >>>>>>>> > >>>>>>>> Lets also work on adding tests to the helm chart to ensure things > >> are > >>>>>>>> going fine. > >>>>>>>> > >>>>>>>> Best, > >>>>>>>> Shubham. > >>>>>>>> > >>>>>>>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma < > >>>>> sharma.ashish...@gmail.com> > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> Hi everyone, > >>>>>>>>> > >>>>>>>>> Update on HELM project:- > >>>>>>>>> > >>>>>>>>> We have added some features to the chart that includes:- > >>>>>>>>> * Making chart configurable with both mysql and postgres database > >>>>>>>>> * We are now using configmaps and secrets for storing database > >>>>>>>> credentials > >>>>>>>>> * We use helm dependency to manage the dependencies of the chart > >>>>>>>>> > >>>>>>>>> I think our chart is ready to be deployed in the incubator > section > >> of > >>>>>>>> helm > >>>>>>>>> community repo. <https://github.com/helm/charts> If you guys > >> agree I > >>>>>>>> would > >>>>>>>>> make the necessary changes required to deploy it there. > >>>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> - Ashish > >>>>>>>>> > >>>>>>>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol < > vinc...@massol.net> > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Hi Ashish, > >>>>>>>>>> > >>>>>>>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma < > >> sharma.ashish...@gmail.com > >>>>>> > >>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Hi everyone, > >>>>>>>>>>> > >>>>>>>>>>> The first week update of helm chart xwiki project. We currently > >>>>> have a > >>>>>>>>>>> chart that would deploy xwiki mysql stateful service with a > >>>>> persistent > >>>>>>>>>>> volume. The link to the repo:- > >>>>>>>>>>> > >>>>>>>>>>> https://github.com/xwiki-contrib/xwiki-helm > >>>>>>>>>> > >>>>>>>>>> Thanks for the update. > >>>>>>>>>> > >>>>>>>>>> Would be great if you could update the README file to explain > who > >> to > >>>>>>>> use > >>>>>>>>>> it. > >>>>>>>>>> > >>>>>>>>>> Thanks > >>>>>>>>>> -Vincent > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Thank You > >>>>>>>>>>> Ashish > >>>>>>>>>>> > >>>>>>>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma < > >>>>>>>>>> sharma.ashish...@gmail.com> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol < > >>>>> vinc...@massol.net> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Hi Ashish, > >>>>>>>>>>>>> > >>>>>>>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma < > >>>>>>>> sharma.ashish...@gmail.com> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi everyone, > >>>>>>>>>>>>>> Thanks for having me here > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> About Me > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I am Ashish Sharma, selected as a student for Google Summer > of > >>>>>>>> Code. I > >>>>>>>>>>>>> am > >>>>>>>>>>>>>> final year student enrolled in Guru Gobind Singh > Indraprastha > >>>>>>>>>>>>> University, > >>>>>>>>>>>>>> Delhi. I am a resident of India. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Profiles > >>>>>>>>>>>>>> GitHub - https://github.com/ashish932/xwiki-helm-chart/ > >>>>>>>>>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/ > >>>>>>>>>>>>>> Riot - @ashish932:matrix.org > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I will be presenting my project "Helm Chart for XWiki" to > all > >> of > >>>>>>>> you. > >>>>>>>>>>>>>> Following > >>>>>>>>>>>>>> are the relevant details. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Helm Chart for XWiki > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Mentors: Shubham Jain, Neha Gupta > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Technologies: Kubernetes, Docker, other if required > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Overview > >>>>>>>>>>>>>> The proposed project is a helm chart that would deploy xwiki > >> as > >>>>>>>> highly > >>>>>>>>>>>>>> available and reliable. It should be configurable with > >> different > >>>>>>>>>>>>>> databases(either a standalone database or a clustered one) > >> that > >>>>> are > >>>>>>>>>>>>>> configurable with xwiki. It would give the option to either > >>>>>>>> configure > >>>>>>>>>>>>> solr > >>>>>>>>>>>>>> externally (standalone or clustered) or managed within the > >>>>>>>> container. > >>>>>>>>>>>>> > >>>>>>>>>>>>> It also needs to be deployed on various Servlet Containers > and > >>>>> with > >>>>>>>> a > >>>>>>>>>>>>> LibreOffice server configured. > >>>>>>>>>>>>> > >>>>>>>>>>>>> In short, it should support all the configurations supported > by > >>>>> the > >>>>>>>>>>>>> official XWiki Docker images :) > >>>>>>>>>>>>> > >>>>>>>>>>>>>> It > >>>>>>>>>>>>>> should deploy the app on a shared file system like a rook. > It > >>>>>>>> should > >>>>>>>>>>>>>> support Istio virtual services, istio matrix, and istio > >>>>> distributed > >>>>>>>>>>>>> tracing > >>>>>>>>>>>>>> and should be a secured system with RBAC and security > >> credential > >>>>>>>>>>>>> rotation. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Ok I have no knowledge about this… The shared file system > >> sounds > >>>>>>>> good > >>>>>>>>>> to > >>>>>>>>>>>>> support clustering XWiki for the permanent directory sharing. > >>>>>>>>>>>>> > >>>>>>>>>>>>>> The chart should be easily deployed on GKE and amazon EKS. > >>>>>>>>>>>>> > >>>>>>>>>>>>> On locally on minikube for testing I guess? > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Yes we can test using minikube one > >>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Features > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> -> Support for different Databases > >>>>>>>>>>>>>> -> Choice between using an external database, a single node > >> DB > >>>>> or > >>>>>>>> a > >>>>>>>>>>>>>> multi-cluster DB setup > >>>>>>>>>>>>>> -> Support for shared file system > >>>>>>>>>>>>>> -> Support for istio and it's services > >>>>>>>>>>>>>> -> RBAC, SSL and other security methods > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> If you have any features in mind that should be added please > >>>>> feel > >>>>>>>> free > >>>>>>>>>>>>> to > >>>>>>>>>>>>>> reply to this mail. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Some Design Questions? > >>>>>>>>>>>>>> -> Which Databases should be supported? > >>>>>>>>>>>>> > >>>>>>>>>>>>> See > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>> > >> > https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy > >>>>>>>>>>>>> > >>>>>>>>>>>>> More generally see the XWiki Docker image documentation and > >>>>>>>>>>>>> > https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy > >>>>>>>>>>>>> > >>>>>>>>>>>>>> -> As we have to detach solr out of the docker container(run > >> it > >>>>> in > >>>>>>>> an > >>>>>>>>>>>>>> independent container) would be there a requirement for a > code > >>>>>>>> change, > >>>>>>>>>>>>> and > >>>>>>>>>>>>>> we should approach it? > >>>>>>>>>>>>> > >>>>>>>>>>>>> See the doc for Solr which explains how to configure an > >> external > >>>>>>>> SOLR. > >>>>>>>>>>>>> There are some XWiki code/config to deploy in the external > SOLR > >>>>>>>> AFAIR. > >>>>>>>>>>>>> comunity controls chart > >>>>>>>>>>>>>> -> Apart from solr is there any other stateful service that > >>>>> could > >>>>>>>> or > >>>>>>>>>>>>> should > >>>>>>>>>>>>>> be detached from the docker container? > >>>>>>>>>>>>> > >>>>>>>>>>>>> Which docker container are you referring to? The Servlet > >>>>> Container > >>>>>>>> one? > >>>>>>>>>>>>> > >>>>>>>>>>>>> xwiki:lts-mysql-tomcat > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>>> Here is my current repository which deploys XWiki for MySQL > >>>>>>>> database > >>>>>>>>>>>>> using > >>>>>>>>>>>>>> official XWiki docker container:- > >>>>>>>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/ > >>>>>>>>>>>>> > >>>>>>>>>>>>> Will need to have a look. Some additional questions: > >>>>>>>>>>>>> > >>>>>>>>>>>>> 1) Is there an official place for publishing helm charts, > >>>>> similar to > >>>>>>>>>>>>> dockerhub for docker images? How do you provide the helm > charts > >>>>> to > >>>>>>>>>> XWiki > >>>>>>>>>>>>> users in term of packaging? In other words, what will we/you > >>>>>>>> document > >>>>>>>>>> on > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>> > >> > https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/ > >>>>>>>>>>>>> ? > >>>>>>>>>>>>> > >>>>>>>>>>>>> Yes, There is a centralized community control chart > management > >>>>>>>>>>>> repository. Helm charts are hosted there. Organizations like > >>>>> mysql, > >>>>>>>>>>>> wordpress uses it. > >>>>>>>>>>>> https://github.com/helm/charts > >>>>>>>>>>>> But in addition to that, we can build our own charts > repository > >>>>>>>> locally, > >>>>>>>>>>>> and host it. Similar to docker run command, we have helm > update > >>>>>>>> command > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>> 2) How do you plan to have smoke tests for the XWiki Helm > >>>>> charts, as > >>>>>>>>>> part > >>>>>>>>>>>>> of the build, to verify that they work fine? How can we > execute > >>>>>>>> XWik’s > >>>>>>>>>>>>> functional tests based on TestContainers using Helm Charts? > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> There are some tools available > >>>>>>>>>>>> chart-testing(started by helm) -98 commits > >>>>>>>>>>>> terratest - 1039 commits (It looks a good tool to use) > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>> 3) Do the helm charts for XWiki need to be built? If so, can > it > >>>>> be > >>>>>>>> done > >>>>>>>>>>>>> with Maven? Since we’ll be offering several I guess, how do > you > >>>>>>>> avoid > >>>>>>>>>> code > >>>>>>>>>>>>> duplication and make maintenance of them as simple as > possible? > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> No, the charts don’t need to be built externally, though it > gets > >>>>>>>>>>>> compressed when pushed f we build to chart repository(this is > an > >>>>>>>>>> automatic > >>>>>>>>>>>> step in case of official chart repo and we can automate using > ci > >>>>> our > >>>>>>>> own > >>>>>>>>>>>> chart repo) > >>>>>>>>>>>> > >>>>>>>>>>>> 4) Could you provide a roadmap with various milestones and > >> dates? > >>>>> See > >>>>>>>>>> the > >>>>>>>>>>>>> last roadmap email I sent on the devs list and please reply > to > >>>>> it. > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Okay I would work on that and update the progress on the mail > >>>>> thread > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>> 5) It seems you missed the bonding period so you’ll need to > >> catch > >>>>>>>> up on > >>>>>>>>>>>>> that. Could you pick some jira issue and work on them > quickly? > >>>>> This > >>>>>>>> is > >>>>>>>>>> to > >>>>>>>>>>>>> make sure you use the proper xwiki communication channels, > >> learn > >>>>> to > >>>>>>>>>> work > >>>>>>>>>>>>> with the xwiki team and start understanding the xwiki > >>>>> architecture, > >>>>>>>>>> and how > >>>>>>>>>>>>> extensions are developed for XWiki. Since you’re a bit late, > >>>>> you’ll > >>>>>>>>>> need to > >>>>>>>>>>>>> catch up and achieve this very quickly now and be present > full > >>>>> time > >>>>>>>> on > >>>>>>>>>> the > >>>>>>>>>>>>> xwiki chat :) > >>>>>>>>>>>>> > >>>>>>>>>>>> I would start doing that > >>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> 6) Idea: please review the issues on the XWiki Docker > project’s > >>>>>>>> jira. > >>>>>>>>>>>>> Maybe you can help with those too as part of > >> onboarding/bonding. > >>>>> It > >>>>>>>>>> should > >>>>>>>>>>>>> also give you additional use cases for the XWiki Helm Charts > >> IMO. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks! I’m eager to follow your work and see xwiki helm > charts > >>>>> in > >>>>>>>>>> action. > >>>>>>>>>>>>> > >>>>>>>>>>>>> -Vincent > >>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thank You > >>>>>>>>>>>>>> Ashish Sharma > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thank You > >>>>>>>>>>>> Ashish > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>> > >>>>> > >> >
