This started as a conversation between me and Scott and Paul Burney about how to deploy the new OTrunk curnit for the TEEMSS2 DIY site to enable learner data persistence. I thought after writing this long response to Scott's suggestions that this work could be consider part of a very very early alpha implementation of the pas researcher pack and that some of the issues could be of interest to a wider group.
background: Scott made an otrunk curnit template described here: http://confluence.concord.org/display/TMS/OTrunk+Curnit I'm using it to add sail persistence to the TEEMSS2 DIY portal: http://teemss2diy.concord.org/ and also adding simple role and scope mechanisms for teachers and class groupings. After the initial reporting for tels is working (early this week) I'll fold that in also. ****************************************************************************** At 2:48 PM -0500 1/28/07, Scott Cytacki wrote: >I answered the questions below. But before going into them. I think >you should try something: >Setup your maven settings. >make sure you .m2/settings.xml has this: > ><settings> > <profiles> > <profile> > <id>concord-profile</id> > <properties> > <concord.storepass>xxxx</concord.storepass> > </properties> > </profile> > </profiles> > <activeProfiles> > <activeProfile>concord-profile</activeProfile> > </activeProfiles> ></settings> > >Check out the pas-jnlp project on your local machine. pas-jnlp is >located in the tels svn under the Pas Suite directory. Once you have >that checked out, go to the folder in the commandline, and run the >following commands: > >export MAVEN_OPTS="-Xmx1024m" >mvn -Dcmd.line.date="`date`" -U >org.telscenter.maven-plugins:webstart-maven-plugin:1.0-alpha-1-SNAPSHOT:jnlp > >(those are a subset of the commands in deploy.sh script) > >If it has errors try running it again and see if it gets further. > >It should create a target/jnlp directory that contains subdirectories >and inside of those all the jar files. If you got the settings.xml file >setup correctly the jars will be signed. It will also pack200 compress >them, which you aren't going to need. > >It will also generate a few jnlp files which reference these jars. >These jars are pulled from our various maven repositories. They should >be the most up-to-date version of the jars. Some could even be broken >because they are built automatically, but I think they will all be >working. Thanks for all that info! It worked though it was awfully involved for what what I was trying to accomplish. I did it in my home dir on otto. It took otto 15 minutes and that's a fast computer with a great network connection. There were a number of errors even after running multiple times. With maven and this particular build process it takes a fair bit of work to just find out what the errors mean and whether they matter. There are of course about a great deal of warnings generated by checking many different repositories over and over for the same things they didn't have the last time they were checked. For example the maven process checked seven repositories for: jsr223/script/1.0ea/script-1.0ea.pom and didn't find it and then went on to warn about a different pom not found. Reading that I'd suppose that the pom (and presumably the jar it refers to) couldn't be found but the jar appears to have been downloaded in the repository 50 minutes earlier (resumably during my first build): 92055 Jan 28 19:26 .m2/repository/jsr223/script/1.0ea/script-1.0ea.jar org/eclipse/emf/ecore-xmi/2.2.0/ecore-xmi-2.2.0.pom org/eclipse/emf/common/2.2.0/common-2.2.0.pom org/eclipse/emf/ecore/2.2.0/ecore-2.2.0.pom OK, I suppose there is a logic here and I could read the maven documentation to find out but I worry that while I could figure it out I'd end up quite annoyed at the design ... Here's a list of errors that maven generates: ******************************maven errors*********************************** Building Common JNLP Parent No resources found in /home/sbannasch/pas-jnlp/common-jnlp-parent/src/main/jnlp skipping project because no main class: null was found Building Pas Basic EMF Post JNLP No resources found in /home/sbannasch/pas-jnlp/common-jnlp-parent/../common-jnlp/../basic-emf-post-snapshot/src/main/jnlp Building PLR Everything Snapshot No resources found in /home/sbannasch/pas-jnlp/common-jnlp-parent/../common-jnlp/../plr-everything-jdic-post-snapshot/src/main/jnlp Building jdic browser test No resources found in /home/sbannasch/pas-jnlp/common-jnlp-parent/../common-jnlp/../jdic-browser-test/src/main/jnlp [WARNING] Something failed while checking if the main class contains the main() method. This is probably due to the limited classpath we have provided to the class loader. The specified main class (Browser.Browser) found in the jar is *assumed* to contain a main method... org/jdesktop/jdic/browser/IWebBrowser Building fs browser test No resources found in /home/sbannasch/pas-jnlp/common-jnlp-parent/../common-jnlp/../fs-browser-test/src/main/jnlp [WARNING] Something failed while checking if the main class contains the main() method. This is probably due to the limited classpath we have provided to the class loader. The specified main class (org.xhtmlrenderer.demo.browser.BrowserStartup) found in the jar is *assumed* to contain a main method... com/jgoodies/looks/plastic/PlasticXPLookAndFeel Building Wise Project Conveter [INFO] task-segment: [org.telscenter.maven-plugins:webstart-maven-plugin:1.0-alpha-1-SNAPSHOT:jnlp] Which ended with this long (and possibly informative) error message: [ERROR] BUILD ERROR [INFO] ------------------------------------------------------------------------ [INFO] Failed to resolve artifact. Missing: ---------- 1) jtidy:jtidy:jar:8.0-SNAPSHOT Try downloading the file manually from the project website. Then, install it using the command: mvn install:install-file -DgroupId=jtidy -DartifactId=jtidy \ -Dversion=8.0-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file Path to dependency: 1) org.telscenter.jnlp:wise-converter:pom:0.1.0-SNAPSHOT 2) org.telscenter:wise-project-converter:jar:0.3.0-SNAPSHOT 3) jtidy:jtidy:jar:8.0-SNAPSHOT ---------- 1 required artifact is missing. for artifact: org.telscenter.jnlp:wise-converter:pom:0.1.0-SNAPSHOT from the specified remote repositories: tels-repo-external_non_free (http://repo.telscenter.org/external_non_free), central (http://repo1.maven.org/maven2), cc-repo-internal-snapshot (http://source.concord.org/maven2/internal_snapshot), codehaus-central (http://repository.codehaus.org), tels-repo-internal (http://repo.telscenter.org/internal/), tels-repo-external_free (http://repo.telscenter.org/external_free), l2fprod (http://www.l2fprod.com/maven2), tels-repo-internal-snapshot (http://repo.telscenter.org/internal_snapshot/), cc-repo-internal (http://source.concord.org/maven2/internal) ***************************end of maven errors******************************** And indeed it was true that jtidy isn't in the repository (just some <metadata> elements in xml files ) ls ~/.m2/repository/jtidy/jtidy/8.0-SNAPSHOT/ maven-metadata-cc-repo-internal-snapshot.xml maven-metadata-codehaus-central.xml maven-metadata-tels-repo-internal-snapshot.xml Well the message are bit hard to interpret ... I'll check the output in target/jnlp: * the jnlps all seem to be there except for the wise project converter (in the build spelled 'conveter') * the jars the jnlps reference also all seem to be there, but ... ** Some directories have jar and jar.pack (ex: org.eclipse.emf), some have jar, jar.pack and jar.pack.gz (org.concord.netlogo), and some have three of most jars but only two of others (org.concord has three forms of every jar except for swing__V0.1.0-20070124.180626-11.jar -- there is no gz variant)?? I might be remembering wrong but I thought that when the webstart servlet delivered pack200 jars to JRE 1.5 and up it always delivered them gzipped -- is there any reason to have the pack200 files around? The basic flow I was imagining is that there are one or more locations where builds happen and jars get versioned and versioned jars get packed and both of these get signed and packed ones get gzipped, and jnlps get made pointing to these jars. I'd like to be able to pull all these resources into a deployment container or dir without rebuilding, signing, or packing the jars and without looking at multiple other dependent repositories on the net (only because these dependencies have already been pulled by the build processes in the deployment locations I am pulling from). And I'd actually like to be able to pull the pertinent jnlps and then pull all the jars (in their variant forms) those jnlps refer to. In the case above I just have the jars the most recent builds of the jnlps refer to. The next time I pulled a copy I'd only grab the new and changed resources (and actually there should only be new resources, the changed items are probably just build scripts and while I need those for the maven building process I don't need them in my simpler pull model). >There is a project analogous to pas-jnlp in CCs CVS: >Projects/MavenJnlp >If you run the same maven command in this one it will download the jars >and make a jnlp for the teemss content. You will have to tweak this >line: ><workDirectory>/web/continuum.concord.org/jnlp</workDirectory> >of the common-jnlp/pom.xml file. That line configures where the jars >will end up. This project is setup to run on CC's continuum. I haven't >run it in a while though. > >The point of this is that the are already systems in place to do what >you want. They are systems but I'm not sure they optimally fit the functions I'd like to perform. I'm looking to script and automate a process that really should only need to happen after building is finished. Check out the Simile projects (they have a bunch other cool projects in addition to Timeline): For example to get a local copy of the Timeline code: http://simile.mit.edu/timeline/ and run it using Jetty there are three steps: svn co http://simile.mit.edu/repository/timeline/trunk timeline/trunk cd timeline/trunk ./run They aren't using maven and of course when I check out some of there other projects they are all using svn externals to get a copies of external dependencies so there are multiple copies of Jetty. Luckily Jetty is reasonably small: ls -lh lib 419K Jan 28 12:39 jetty-6.0.1.jar 104K Jan 28 12:39 jetty-util-6.0.1.jar 133K Jan 28 12:39 servlet-api-2.5-6.0.1.jar 15K Jan 28 12:39 start.jar >On Sun, 2007-01-28 at 11:57 -0500, Stephen Bannasch wrote: >> What is the minimal set of sail jars needed to run the teemss2 content? >Don't know right now. > >> Is there a minimal sail jnlp I can use as a template? >Nope, because the current oturnk curnit support is entangled with pas That's too bad -- I can just hear myself telling Carolyn that she really doesn't want to know why the TEEMSS software is now a 20 MB download but not to worry because it won't be for long ;-) >at the moment. This will be separate soon, That will be a very good thing. I can easily imagine this getting to be a problem at her workshop in Alaska next week when everybody tries to start at the same time. If she associates this with SAIL it might leave a lasting impression I'd really like to avoid. When I implement some new functionality (learner data persistence for teemss2) that I think non-technical people would appreciate I would *VERY MUCH* like to avoid making something else more painful for them. >so in the mean time just use >all the jars from plr-snapshot-jdic jnlp. sigh ... I hope not for long. > > Where can I find updated otrunk jars? >They are built by cc's continuum server, and deployed to cc's maven >repository. The portfolio and sensor jars are not currently being built >this way. So is the new popup functionality in otrunk__V0.1.0-20070128.000052-14.jar? > > Are there newer teemss2 jars? >I'm not sure the distiction here between teemss2 and otrunk. If by >teemss2 you mean the portfolio jar and the sensor jars then no there are >not newer ones yet. Yes, these are what I was referring to and I didn't know if you or Shengyao had made any updates. > > I'd like a webstart servlet running at cc and a way to deploy versioned, > > released and pack200'ed jars there. >It is in the works. We just need a tomcat instance setup and a apache >proxy pointed at it. I can't remember but I think Paul might have >already done this. Then we just deploy the war which is built by >pas-jnlp. When I ran the maven command you referenced it didn't make a war -- probably need to look in deploy.sh for the extra steps. > > But for the diy site right now I just want static copies of the > > appropriate jars on the same server. If you give me the references I >> can collect them in one place. I don't want to rely on the tels server >> for delivering the diy stuff. >If you can run the pas-jnlp project described above then you will have >all the jars you need. They will be collected in one place. I certainly do have lots of jars -- but if I wanted to run Jetty proxied from Apache with the webstart servlet I don't yet have all the files I need (there are some missing gzipped jars). > > I want to be able to easily collect jars from multiple locations and > > put sets of these on different computers, for ex: the diy server >> (otto), or my local computer for either direct local use or for >> serving as static http resources or more dynamically via the webstart >> servlet or equivalent. If I'm deploying from a cc server I'd probably >> like to get copies of the updated jars whenever they are deployed >> elsewhere (either notify and pull, or push). For my local computer I >> would instead be a pull operation. >This is what the pas-jnlp and MavenJnlp projects do. They are pull. >They pull the jars from maven repositories, and sign them. If we want >this widely deployed we are going to have to figure out about the >signing, because we don't want everybody using the CC signing cert. I suppose that's what jnlp extensions are for. I wonder what actually happens when the jws client gets a signed set of jars. I suppose it might have to make a request to the place we got our cert from to check it -- and if there were many jnlp extensions there might be many different requests to check the signatures. Hmmm ... I wonder what we mean when we say 'widely deployed'. One scenario is the pas researcher pack and I could imagine it would be nice if it worked like this: svn co http://tels.svn.sourceforge.net/svntrunk/pas-researcher-pack/trunk pas-reseacher-pack cd pas-reseacher-pack ./run And now they have their own portal, sds, sps, reporting-app, pedagogic and technical and examples to run, play with , and generate data for reports with, developer doc to dig into, source code to change etc. An example to compare with is the Open Lazlo project (large Java project). They make a very large download available (50 MB): http://www.openlaszlo.org/download It's very easy to get started after downloading and it has has all sorts of great stuff to play with and try out. They also make the source available via subversion and describe ways to checkout and build stable releases that have passed QA or trunk. While they have documented the checkout and build process quite well it is by no means simple: http://wiki.openlaszlo.org/SubversionBuildInstructions One key difference however (it's at least part of our vision) is that both the artifacts programmers make and the artifacts authors make can be easily distributed and shared. What happens when they make a new cool component and want to share it (or to get updates of the cool new things we've added. I'd love to see some examples of successful projects hat do this (if there are any). In these varying situations what roles and in what situations should artifacts be signed and who needs to trust whom? > > Tell me if I have this right: When you deploy the war file on tels you > > are including complete copies of all the versions of all the jars and >> then the jar differencing and pack200 compression happens on the fly >> by the webstart servlet in response to requests? >Yes that is correct. Actually it appears that you are generating: * a versioned jar * a packed versioned jar * and in some cases a gzipped packed versioned jar and this is what is delivered to the webstart servlet. > > Could I just copy the war file or the directories of jars it expands >> into onto otto? Could I use rsync (one-way synch) to do this and >> collect jars from multiple locations? >You could, but you should try using the pas-jnlp and MavenJnlp projects >first. These will collect the jars from multiple maven repositories. I've still got to try the CC MavenJnlp step. That's next. >If you can't get the maven projects to work, just log into tels-develop >and look at the /var/local/maven-jnlp-staging/pas-jnlp folder. Inside >of there you will find the target/jnlp which has all the jars you want. >You could use rsync over ssh to pull down all those jars, and keep them >updated on your system. Most of the teemss jars are included the only >ones that should be missing are the sensor jars, and the portfolio jar. Great. -- - Stephen Bannasch Concord Consortium, http://www.concord.org --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "SAIL-Dev" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/SAIL-Dev?hl=en -~----------~----~----~----~------~----~------~--~---
