Created CONNECTORS-1049 to describe relocating plugins and connector processes. Karl
On Fri, Sep 26, 2014 at 5:28 AM, Karl Wright <[email protected]> wrote: > I've replied to the jetty vs. tomcat issue in the CONNECTORS-345 comment > thread. > Thanks! > Karl > > On Fri, Sep 26, 2014 at 5:22 AM, Karl Wright <[email protected]> wrote: > >> I've moved CONNECTORS-345 to "Fix in 2.0" and attached your comments. >> Let's talk about the jetty deployment issues in that ticket thread. >> >> Karl >> >> >> On Fri, Sep 26, 2014 at 5:16 AM, Karl Wright <[email protected]> wrote: >> >>> CONNECTORS-1048 created for including jar versions. >>> Karl >>> >>> On Fri, Sep 26, 2014 at 5:03 AM, Karl Wright <[email protected]> wrote: >>> >>>> Hi Abe-san, >>>> >>>> It seems to me that we are discussing some distinct and independent >>>> improvements, so maybe I will open specific tickets for each one and we can >>>> work on things one at a time. But, first: >>>> >>>> bq. I just simplified directories since other Apache projects look like >>>> much simpler. >>>> >>>> That's true, but I don't know of any other Apache project which >>>> attempts to handle conditional compilation to the degree that we have to in >>>> ManifoldCF. So I think we should expect things to be more complicated. :-) >>>> >>>> bq. BTW, for a few jars(oracle,mysql,alfresco,jcifs,livelink,jtds and >>>> other), to create *-proprietary directories is awkward, isn't it? >>>> >>>> I've never particularly liked the -proprietary paradigm, but I could >>>> think of no other way. The reasons behind it was that we *must* legally >>>> exclude anything proprietary from the distribution. By making sure that >>>> every proprietary jar is in a well-labeled directory, I can simply make >>>> sure all such directories are excluded. But that also includes wars. >>>> Since wars must include all jars, that meant that the build had to create >>>> two different sets of wars, and put them in different directories. The >>>> only other alternative is to guarantee that we always build our release >>>> candidates from a fresh checkout each time, and that seemed risky. >>>> >>>> I agree that it would be more convenient for the user if our build >>>> created one set of wars based on what was included by the user, and that we >>>> shipped just the minimal binary that didn't include proprietary code. If >>>> we can solve the war problem, we can do that. >>>> >>>> Karl >>>> >>>> >>>> >>>> On Fri, Sep 26, 2014 at 12:51 AM, Shinichiro Abe < >>>> [email protected]> wrote: >>>> >>>>> Hi Karl, >>>>> >>>>> Thanks for the reply. >>>>> >>>>> > (1) I don't understand what you mean by, "because build.xml in >>>>> framework is >>>>> > currently too long to include class >>>>> > paths in lib". Can you clarify? >>>>> >>>>> Rerated to (5) , manifest-cp configuration is not cool at all. >>>>> >>>>> https://svn.apache.org/viewvc/manifoldcf/trunk/framework/build.xml?view=markup#l1343 >>>>> If new jars introduced with new feature, manifest-cp-proprietary must >>>>> be tweaked, too. >>>>> (i18n properties files also bother us.) >>>>> >>>>> Alternatively, the ant task below is useful. >>>>> >>>>> common-build.xml: Add artifact-version to dest. >>>>> + <get >>>>> src="${maven-base-url}/${project-path}/${artifact-name}/${artifact-version}/${artifact-name}-${artifact-version}.${artifact-type}" >>>>> dest="${target}/${artifact-name}-${artifact-version}.${artifact-type}"/> >>>>> >>>>> framework/build.xml: New manifest-cp property setting works fine. >>>>> >>>>> - <property name="manifest-cp-74" value="${manifest-cp-73} >>>>> ../lib/xmlsec.jar"/> >>>>> - <property name="manifest-cp-75" value="${manifest-cp-74} >>>>> ../lib/opensaml.jar"/> >>>>> >>>>> + <property name="liblocation" location="../lib" /> >>>>> + <path id="lib-jars"> >>>>> + <fileset dir="${liblocation}" includes="*.jar"/> >>>>> + </path> >>>>> + <pathconvert property="manifest-cp" refid="lib-jars" >>>>> targetos="unix" pathsep=" "> >>>>> + <map from="${liblocation}" to="../lib"/> >>>>> + <map from="\" to="/"/> >>>>> + </pathconvert> >>>>> >>>>> + <!-- <property name="manifest-cp" value="${manifest-cp-75}"/> >>>>> --> >>>>> >>>>> > (2) Some of these suggestions seem to be making distinctions between >>>>> files >>>>> > and directories that I don't understand the reason for. >>>>> >>>>> 1. I came to mind CONNECTORS-345. >>>>> 2. Currently I have to create scripts in production: >>>>> startup.sh -- >>>>> java -jar start.jar >>>>> echo $! > mcf.pid >>>>> shutdown.sh -- >>>>> kill cat(mcf.pid) <-- MCF's embedded Jetty example can't call >>>>> agent-shutdown AFAIK but this script isn't cool. >>>>> >>>>> 3. start.jar command line options are useful. >>>>> >>>>> http://www.eclipse.org/jetty/documentation/current/start-jar.html#d0e8418 >>>>> >>>>> 4. If we can separate Jetty-lib and mcf-lib, we could deploy Tomcat. >>>>> Of course combined.war can do that though. >>>>> I thought additional jars could copy to another directory, and might >>>>> load these using Custom ClassLoader. >>>>> >>>>> http://www.eclipse.org/jetty/documentation/current/jetty-classloading.html >>>>> >>>>> 5. But If users don't want to config jetty.xml(e.g. thread pool, >>>>> requestHeaderSize and so on in jetty.xml -- >>>>> http://wiki.eclipse.org/Jetty/Howto/Configure_Connectors), >>>>> I will drop these suggestions. >>>>> >>>>> > (3) I agree with "connector-specific-processes" and "plugins". But >>>>> other >>>>> > hierarchy additions seem less helpful, such as hiding the examples >>>>> under >>>>> > additional levels of hierarchy. I think it should be possible >>>>> immediately >>>>> > for a user to see what examples are available. But maybe we could >>>>> change >>>>> > the names to be clearer. >>>>> Yes. Especially I want to change name *-proprietary into something >>>>> like *-extra because "proprietary".length() is long. >>>>> >>>>> BTW, for a few jars(oracle,mysql,alfresco,jcifs,livelink,jtds and >>>>> other), to create *-proprietary directories is awkward, isn't it? >>>>> I know CONNECTORS-402 intent, but more simplified, can we say that "if >>>>> you use these jars, >>>>> please get source and run ''ant make-deps build"? -- I usually say so, >>>>> no problems occur for now. >>>>> >>>>> > (4) Rather than group together xxx and xxx-proprietary, and yyy and >>>>> > yyy-proprietary, it would be more appropriate to have a "proprietary" >>>>> > directory and an "open" directory", with xxx and yyy under them. >>>>> >>>>> Ok. >>>>> But I want to know, can we deprecate example and move >>>>> example-proprietary to example? >>>>> I'd like to merge these examples. If proprietary jars exists, MCF just >>>>> have to load from extra-dir. >>>>> I assume that we can have extra-lib which is optional, If it exists, >>>>> MCF will load any Jars >>>>> found in this directory and use them to resolve connectors specified >>>>> in connectors.xml. >>>>> (Solr has plugin class loader at $SOLR_HOME/lib) >>>>> >>>>> I just simplified directories since other Apache projects look like >>>>> much simpler. >>>>> >>>>> Regards, >>>>> Shinichiro Abe. >>>>> >>>>> On 2014/09/26, at 3:01, Karl Wright <[email protected]> wrote: >>>>> >>>>> > Hi Abe-san, >>>>> > >>>>> > Some comments: >>>>> > >>>>> > (1) I don't understand what you mean by, "because build.xml in >>>>> framework is >>>>> > currently too long to include class >>>>> > paths in lib". Can you clarify? >>>>> > >>>>> > (2) Some of these suggestions seem to be making distinctions between >>>>> files >>>>> > and directories that I don't understand the reason for. For >>>>> example: "in >>>>> > process-single directory we can >>>>> > create lib dir(for jetty jars), lib/ext dir(for logger jars), >>>>> resource >>>>> > dir(for logging.ini) and etc dir(for jetty.xml)." Why separate jars >>>>> in >>>>> > this way? They are all necessary at the root level to run the >>>>> example. I >>>>> > would not understand where to add a new jar with this arrangement >>>>> because >>>>> > the meaning of the directories is unclear. >>>>> > >>>>> > (3) I agree with "connector-specific-processes" and "plugins". But >>>>> other >>>>> > hierarchy additions seem less helpful, such as hiding the examples >>>>> under >>>>> > additional levels of hierarchy. I think it should be possible >>>>> immediately >>>>> > for a user to see what examples are available. But maybe we could >>>>> change >>>>> > the names to be clearer. >>>>> > >>>>> > (4) Rather than group together xxx and xxx-proprietary, and yyy and >>>>> > yyy-proprietary, it would be more appropriate to have a "proprietary" >>>>> > directory and an "open" directory", with xxx and yyy under them. >>>>> > >>>>> > (5) Putting version numbers on jars is difficult in some cases, >>>>> especially >>>>> > in construction of start.jar, because the ant methods for >>>>> constructing >>>>> > start.jar are poor. The version of each jar would need to be defined >>>>> > globally in the ant build, and included whenever the jar is >>>>> referenced. >>>>> > >>>>> > Karl >>>>> >>>>> >>>> >>> >> >
