RE: Some more thoughts on a classloader plug-in style architecture
People who write connectors for LCF before this change occurs will need to revamp them completely afterwards. That is, unless they contribute them to the LCF code stream before the work begins... Second, since there is no Q/A infrastructure yet, there's a very good chance that stuff will get busted by such a massive change and go undetected. I would therefore greatly prefer that Q/A infrastructure get created before such changes, but that is not something I have any control over, unfortunately. I'm working as hard as I can towards having at least some core framework tests in place, but the UI testing will necessarily involve many instances of proprietary third-party repositories. Karl -Original Message- From: ext Jack Krupansky [mailto:jack.krupan...@lucidimagination.com] Sent: Wednesday, June 02, 2010 11:06 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Sounds good to me, assuming that LCF remains relatively stable. I am presuming that a fair number of people can and will be using LCF for various purposes well before actual formal "first release" anyway. The point being that delaying the formal release shouldn't slow most people from using LCF as is, provided that any phases of instability are relatively contained, which seems to be the normal case here anyway. -- Jack Krupansky -- From: Sent: Wednesday, June 02, 2010 9:00 AM To: Subject: RE: Some more thoughts on a classloader plug-in style architecture > I've entered a ticket CONNECTORS-40 for this work. What I propose is that > this gets done before first official LCF release, because of the potential > backwards-compatibility issues involved. It is, however, quite a heavy > lift - I can't imagine getting it done in less than a couple of weeks > straight-out effort. To minimize the elapsed development time of this > step, I propose that we not attempt to convert any JSP's from the > framework itself in this round, but merely those for each connector. I'll > start by adding appropriate placeholder methods to the interfaces and all > connectors, and then try to port the UI components one at a time, for one > specific connector. Then, we can have a back-and-forth about how to > refine that one implementation before I attempt the rest. > > Sound reasonable? > > Karl > > > -Original Message- > From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] > Sent: Wednesday, June 02, 2010 8:51 AM > To: connectors-dev@incubator.apache.org > Subject: Re: Some more thoughts on a classloader plug-in style > architecture > > Yeah, definitely embedded Java (ala JSP's <%%> stuff) isn't how > Velocity works. It can call Java code just fine via "tools" (Java > objects) that are injected into the Velocity context. Any > sophisticated business logic can be distilled from the existing plugin > JSPs and migrated to Java classes and added to the Velocity context. > > If specific connectors need more logic, they could ship with their own > tools, and we build in a mechanism to pull in custom tools into the > context. > > Erik > > On Jun 2, 2010, at 8:28 AM, > wrote: > >> I've just spelunked through what I could find online, and it seems >> at least plausible to use Velocity for various LCF HTML templating >> needs. The major concern that I have is that the mix of inline java >> to HTML in the LCF stuff is weighted heavily towards inline java - >> which doesn't seem to be where Velocity's sweet spot is. ;-) The >> general underlying idea of invoking a connector-API-based method >> instead of providing a JSP seems sound, however, no matter what the >> templating engine is that does the final assembly. >> >> Karl >> >> >> -Original Message- >> From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] >> Sent: Wednesday, June 02, 2010 8:19 AM >> To: connectors-dev@incubator.apache.org >> Subject: Re: Some more thoughts on a classloader plug-in style >> architecture >> >> Velocity is just a simple templating engine, so it could be used in >> the intermediate fashion to produce only snippets that mesh into the >> rest of the built-in UI, no problem. >> >> Erik >> >> >> >> On Jun 2, 2010, at 8:04 AM, >> >> wrote: >>> Does the entire UI have to be converted to Velocity for this >>> approach to work? There's an intermediate path that would involve >>> converting only the connector portions, which might be viable. >>> >>> Karl >>> >>> >>> -O
Re: Some more thoughts on a classloader plug-in style architecture
Sounds good to me, assuming that LCF remains relatively stable. I am presuming that a fair number of people can and will be using LCF for various purposes well before actual formal "first release" anyway. The point being that delaying the formal release shouldn't slow most people from using LCF as is, provided that any phases of instability are relatively contained, which seems to be the normal case here anyway. -- Jack Krupansky -- From: Sent: Wednesday, June 02, 2010 9:00 AM To: Subject: RE: Some more thoughts on a classloader plug-in style architecture I've entered a ticket CONNECTORS-40 for this work. What I propose is that this gets done before first official LCF release, because of the potential backwards-compatibility issues involved. It is, however, quite a heavy lift - I can't imagine getting it done in less than a couple of weeks straight-out effort. To minimize the elapsed development time of this step, I propose that we not attempt to convert any JSP's from the framework itself in this round, but merely those for each connector. I'll start by adding appropriate placeholder methods to the interfaces and all connectors, and then try to port the UI components one at a time, for one specific connector. Then, we can have a back-and-forth about how to refine that one implementation before I attempt the rest. Sound reasonable? Karl -Original Message- From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Wednesday, June 02, 2010 8:51 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Yeah, definitely embedded Java (ala JSP's <%%> stuff) isn't how Velocity works. It can call Java code just fine via "tools" (Java objects) that are injected into the Velocity context. Any sophisticated business logic can be distilled from the existing plugin JSPs and migrated to Java classes and added to the Velocity context. If specific connectors need more logic, they could ship with their own tools, and we build in a mechanism to pull in custom tools into the context. Erik On Jun 2, 2010, at 8:28 AM, wrote: I've just spelunked through what I could find online, and it seems at least plausible to use Velocity for various LCF HTML templating needs. The major concern that I have is that the mix of inline java to HTML in the LCF stuff is weighted heavily towards inline java - which doesn't seem to be where Velocity's sweet spot is. ;-) The general underlying idea of invoking a connector-API-based method instead of providing a JSP seems sound, however, no matter what the templating engine is that does the final assembly. Karl -Original Message- From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Wednesday, June 02, 2010 8:19 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Velocity is just a simple templating engine, so it could be used in the intermediate fashion to produce only snippets that mesh into the rest of the built-in UI, no problem. Erik On Jun 2, 2010, at 8:04 AM, wrote: Does the entire UI have to be converted to Velocity for this approach to work? There's an intermediate path that would involve converting only the connector portions, which might be viable. Karl -Original Message- From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Wednesday, June 02, 2010 7:27 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Actually the problem is quite tractable. We switch the UI over to Velocity templates (like Solr's VelocityResponseWriter, for example) and embed the UI bits into plugin JAR files that can live externally. JSPs aren't really workable in this fashion. Velocity templates can be loaded from the file system, the classpath, from a String, or from thin air. For example, VelocityResponseWriter allows templates to load from the actual request (clients can send in a template as an HTTP parameter), if not found it looks on the filesystem, and if not found it looks in the classpath. Erik On Jun 2, 2010, at 6:55 AM, Jack Krupansky wrote: Good point. LCF is a bit more complex than Solr in that sense. Maybe a separate class is needed that has methods to retrieve the crawl and UI components of a connector. Or a small XML file with whatever info about the connector is needed. Or maybe it is simple enough for a properties file. Or maybe just a naming convention so that the name of the UI component can be deduced given the logical name of a connector. -- Jack Krupansky ---------- From: Sent: Wednesday, June 02, 2010 6:45 AM To: Subject: Some more thoughts on a classloader plug-in style architecture It occurred to
Re: Some more thoughts on a classloader plug-in style architecture
Totally reasonable. We'll try to devote some cycles to this too, of course, but probably not for a couple of weeks. Erik On Jun 2, 2010, at 9:00 AM, > wrote: I've entered a ticket CONNECTORS-40 for this work. What I propose is that this gets done before first official LCF release, because of the potential backwards-compatibility issues involved. It is, however, quite a heavy lift - I can't imagine getting it done in less than a couple of weeks straight-out effort. To minimize the elapsed development time of this step, I propose that we not attempt to convert any JSP's from the framework itself in this round, but merely those for each connector. I'll start by adding appropriate placeholder methods to the interfaces and all connectors, and then try to port the UI components one at a time, for one specific connector. Then, we can have a back-and-forth about how to refine that one implementation before I attempt the rest. Sound reasonable? Karl -Original Message- From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Wednesday, June 02, 2010 8:51 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Yeah, definitely embedded Java (ala JSP's <%%> stuff) isn't how Velocity works. It can call Java code just fine via "tools" (Java objects) that are injected into the Velocity context. Any sophisticated business logic can be distilled from the existing plugin JSPs and migrated to Java classes and added to the Velocity context. If specific connectors need more logic, they could ship with their own tools, and we build in a mechanism to pull in custom tools into the context. Erik On Jun 2, 2010, at 8:28 AM, wrote: I've just spelunked through what I could find online, and it seems at least plausible to use Velocity for various LCF HTML templating needs. The major concern that I have is that the mix of inline java to HTML in the LCF stuff is weighted heavily towards inline java - which doesn't seem to be where Velocity's sweet spot is. ;-) The general underlying idea of invoking a connector-API-based method instead of providing a JSP seems sound, however, no matter what the templating engine is that does the final assembly. Karl -Original Message- From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Wednesday, June 02, 2010 8:19 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Velocity is just a simple templating engine, so it could be used in the intermediate fashion to produce only snippets that mesh into the rest of the built-in UI, no problem. Erik On Jun 2, 2010, at 8:04 AM, wrote: Does the entire UI have to be converted to Velocity for this approach to work? There's an intermediate path that would involve converting only the connector portions, which might be viable. Karl -Original Message- From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Wednesday, June 02, 2010 7:27 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Actually the problem is quite tractable. We switch the UI over to Velocity templates (like Solr's VelocityResponseWriter, for example) and embed the UI bits into plugin JAR files that can live externally. JSPs aren't really workable in this fashion. Velocity templates can be loaded from the file system, the classpath, from a String, or from thin air. For example, VelocityResponseWriter allows templates to load from the actual request (clients can send in a template as an HTTP parameter), if not found it looks on the filesystem, and if not found it looks in the classpath. Erik On Jun 2, 2010, at 6:55 AM, Jack Krupansky wrote: Good point. LCF is a bit more complex than Solr in that sense. Maybe a separate class is needed that has methods to retrieve the crawl and UI components of a connector. Or a small XML file with whatever info about the connector is needed. Or maybe it is simple enough for a properties file. Or maybe just a naming convention so that the name of the UI component can be deduced given the logical name of a connector. -- Jack Krupansky ---------- From: Sent: Wednesday, June 02, 2010 6:45 AM To: Subject: Some more thoughts on a classloader plug-in style architecture It occurred to me that a classloader plug-in reader for LCF would not achieve the goal of allowing a fully prebuilt LCF with connector add-ons. The reason, which should have been obvious from the beginning, is because each connector consists not only of the Java implementation, but also a UI component. The UI component will need a mechanism similar to the classloader one in order for everything to work. It is possible
RE: Some more thoughts on a classloader plug-in style architecture
I've entered a ticket CONNECTORS-40 for this work. What I propose is that this gets done before first official LCF release, because of the potential backwards-compatibility issues involved. It is, however, quite a heavy lift - I can't imagine getting it done in less than a couple of weeks straight-out effort. To minimize the elapsed development time of this step, I propose that we not attempt to convert any JSP's from the framework itself in this round, but merely those for each connector. I'll start by adding appropriate placeholder methods to the interfaces and all connectors, and then try to port the UI components one at a time, for one specific connector. Then, we can have a back-and-forth about how to refine that one implementation before I attempt the rest. Sound reasonable? Karl -Original Message- From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Wednesday, June 02, 2010 8:51 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Yeah, definitely embedded Java (ala JSP's <%%> stuff) isn't how Velocity works. It can call Java code just fine via "tools" (Java objects) that are injected into the Velocity context. Any sophisticated business logic can be distilled from the existing plugin JSPs and migrated to Java classes and added to the Velocity context. If specific connectors need more logic, they could ship with their own tools, and we build in a mechanism to pull in custom tools into the context. Erik On Jun 2, 2010, at 8:28 AM, wrote: > I've just spelunked through what I could find online, and it seems > at least plausible to use Velocity for various LCF HTML templating > needs. The major concern that I have is that the mix of inline java > to HTML in the LCF stuff is weighted heavily towards inline java - > which doesn't seem to be where Velocity's sweet spot is. ;-) The > general underlying idea of invoking a connector-API-based method > instead of providing a JSP seems sound, however, no matter what the > templating engine is that does the final assembly. > > Karl > > > -Original Message- > From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] > Sent: Wednesday, June 02, 2010 8:19 AM > To: connectors-dev@incubator.apache.org > Subject: Re: Some more thoughts on a classloader plug-in style > architecture > > Velocity is just a simple templating engine, so it could be used in > the intermediate fashion to produce only snippets that mesh into the > rest of the built-in UI, no problem. > > Erik > > > > On Jun 2, 2010, at 8:04 AM, > wrote: >> Does the entire UI have to be converted to Velocity for this >> approach to work? There's an intermediate path that would involve >> converting only the connector portions, which might be viable. >> >> Karl >> >> >> -----Original Message- >> From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] >> Sent: Wednesday, June 02, 2010 7:27 AM >> To: connectors-dev@incubator.apache.org >> Subject: Re: Some more thoughts on a classloader plug-in style >> architecture >> >> Actually the problem is quite tractable. We switch the UI over to >> Velocity templates (like Solr's VelocityResponseWriter, for example) >> and embed the UI bits into plugin JAR files that can live externally. >> >> JSPs aren't really workable in this fashion. >> >> Velocity templates can be loaded from the file system, the classpath, >> from a String, or from thin air. For example, VelocityResponseWriter >> allows templates to load from the actual request (clients can send in >> a template as an HTTP parameter), if not found it looks on the >> filesystem, and if not found it looks in the classpath. >> >> Erik >> >> On Jun 2, 2010, at 6:55 AM, Jack Krupansky wrote: >> >>> Good point. LCF is a bit more complex than Solr in that sense. >>> >>> Maybe a separate class is needed that has methods to retrieve the >>> crawl and UI components of a connector. >>> >>> Or a small XML file with whatever info about the connector is >>> needed. Or maybe it is simple enough for a properties file. >>> >>> Or maybe just a naming convention so that the name of the UI >>> component can be deduced given the logical name of a connector. >>> >>> -- Jack Krupansky >>> >>> -- >>> From: >>> Sent: Wednesday, June 02, 2010 6:45 AM >>> To: >>> Subject: Some more thoughts on a classloader plug-in style
Re: Some more thoughts on a classloader plug-in style architecture
Yeah, definitely embedded Java (ala JSP's <%%> stuff) isn't how Velocity works. It can call Java code just fine via "tools" (Java objects) that are injected into the Velocity context. Any sophisticated business logic can be distilled from the existing plugin JSPs and migrated to Java classes and added to the Velocity context. If specific connectors need more logic, they could ship with their own tools, and we build in a mechanism to pull in custom tools into the context. Erik On Jun 2, 2010, at 8:28 AM, > wrote: I've just spelunked through what I could find online, and it seems at least plausible to use Velocity for various LCF HTML templating needs. The major concern that I have is that the mix of inline java to HTML in the LCF stuff is weighted heavily towards inline java - which doesn't seem to be where Velocity's sweet spot is. ;-) The general underlying idea of invoking a connector-API-based method instead of providing a JSP seems sound, however, no matter what the templating engine is that does the final assembly. Karl -Original Message- From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Wednesday, June 02, 2010 8:19 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Velocity is just a simple templating engine, so it could be used in the intermediate fashion to produce only snippets that mesh into the rest of the built-in UI, no problem. Erik On Jun 2, 2010, at 8:04 AM, wrote: Does the entire UI have to be converted to Velocity for this approach to work? There's an intermediate path that would involve converting only the connector portions, which might be viable. Karl -Original Message- From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Wednesday, June 02, 2010 7:27 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Actually the problem is quite tractable. We switch the UI over to Velocity templates (like Solr's VelocityResponseWriter, for example) and embed the UI bits into plugin JAR files that can live externally. JSPs aren't really workable in this fashion. Velocity templates can be loaded from the file system, the classpath, from a String, or from thin air. For example, VelocityResponseWriter allows templates to load from the actual request (clients can send in a template as an HTTP parameter), if not found it looks on the filesystem, and if not found it looks in the classpath. Erik On Jun 2, 2010, at 6:55 AM, Jack Krupansky wrote: Good point. LCF is a bit more complex than Solr in that sense. Maybe a separate class is needed that has methods to retrieve the crawl and UI components of a connector. Or a small XML file with whatever info about the connector is needed. Or maybe it is simple enough for a properties file. Or maybe just a naming convention so that the name of the UI component can be deduced given the logical name of a connector. -- Jack Krupansky -- From: Sent: Wednesday, June 02, 2010 6:45 AM To: Subject: Some more thoughts on a classloader plug-in style architecture It occurred to me that a classloader plug-in reader for LCF would not achieve the goal of allowing a fully prebuilt LCF with connector add-ons. The reason, which should have been obvious from the beginning, is because each connector consists not only of the Java implementation, but also a UI component. The UI component will need a mechanism similar to the classloader one in order for everything to work. It is possible, I suppose, for precompiled JSP's to be class-loaded instead of uncompiled JSP's in the lcf-crawler-ui.war file. However this is going to require some care and finesse (and example build.xml files) to get it to work properly. Karl
RE: Some more thoughts on a classloader plug-in style architecture
I've just spelunked through what I could find online, and it seems at least plausible to use Velocity for various LCF HTML templating needs. The major concern that I have is that the mix of inline java to HTML in the LCF stuff is weighted heavily towards inline java - which doesn't seem to be where Velocity's sweet spot is. ;-) The general underlying idea of invoking a connector-API-based method instead of providing a JSP seems sound, however, no matter what the templating engine is that does the final assembly. Karl -Original Message- From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Wednesday, June 02, 2010 8:19 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Velocity is just a simple templating engine, so it could be used in the intermediate fashion to produce only snippets that mesh into the rest of the built-in UI, no problem. Erik On Jun 2, 2010, at 8:04 AM, wrote: > Does the entire UI have to be converted to Velocity for this > approach to work? There's an intermediate path that would involve > converting only the connector portions, which might be viable. > > Karl > > > -Original Message- > From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] > Sent: Wednesday, June 02, 2010 7:27 AM > To: connectors-dev@incubator.apache.org > Subject: Re: Some more thoughts on a classloader plug-in style > architecture > > Actually the problem is quite tractable. We switch the UI over to > Velocity templates (like Solr's VelocityResponseWriter, for example) > and embed the UI bits into plugin JAR files that can live externally. > > JSPs aren't really workable in this fashion. > > Velocity templates can be loaded from the file system, the classpath, > from a String, or from thin air. For example, VelocityResponseWriter > allows templates to load from the actual request (clients can send in > a template as an HTTP parameter), if not found it looks on the > filesystem, and if not found it looks in the classpath. > > Erik > > On Jun 2, 2010, at 6:55 AM, Jack Krupansky wrote: > >> Good point. LCF is a bit more complex than Solr in that sense. >> >> Maybe a separate class is needed that has methods to retrieve the >> crawl and UI components of a connector. >> >> Or a small XML file with whatever info about the connector is >> needed. Or maybe it is simple enough for a properties file. >> >> Or maybe just a naming convention so that the name of the UI >> component can be deduced given the logical name of a connector. >> >> -- Jack Krupansky >> >> -- >> From: >> Sent: Wednesday, June 02, 2010 6:45 AM >> To: >> Subject: Some more thoughts on a classloader plug-in style >> architecture >> >>> It occurred to me that a classloader plug-in reader for LCF would >>> not achieve the goal of allowing a fully prebuilt LCF with >>> connector add-ons. The reason, which should have been obvious from >>> the beginning, is because each connector consists not only of the >>> Java implementation, but also a UI component. The UI component >>> will need a mechanism similar to the classloader one in order for >>> everything to work. >>> >>> It is possible, I suppose, for precompiled JSP's to be class-loaded >>> instead of uncompiled JSP's in the lcf-crawler-ui.war file. >>> However this is going to require some care and finesse (and example >>> build.xml files) to get it to work properly. >>> >>> Karl >>> >>> >
Re: Some more thoughts on a classloader plug-in style architecture
Velocity is just a simple templating engine, so it could be used in the intermediate fashion to produce only snippets that mesh into the rest of the built-in UI, no problem. Erik On Jun 2, 2010, at 8:04 AM, > wrote: Does the entire UI have to be converted to Velocity for this approach to work? There's an intermediate path that would involve converting only the connector portions, which might be viable. Karl -Original Message- From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Wednesday, June 02, 2010 7:27 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Actually the problem is quite tractable. We switch the UI over to Velocity templates (like Solr's VelocityResponseWriter, for example) and embed the UI bits into plugin JAR files that can live externally. JSPs aren't really workable in this fashion. Velocity templates can be loaded from the file system, the classpath, from a String, or from thin air. For example, VelocityResponseWriter allows templates to load from the actual request (clients can send in a template as an HTTP parameter), if not found it looks on the filesystem, and if not found it looks in the classpath. Erik On Jun 2, 2010, at 6:55 AM, Jack Krupansky wrote: Good point. LCF is a bit more complex than Solr in that sense. Maybe a separate class is needed that has methods to retrieve the crawl and UI components of a connector. Or a small XML file with whatever info about the connector is needed. Or maybe it is simple enough for a properties file. Or maybe just a naming convention so that the name of the UI component can be deduced given the logical name of a connector. -- Jack Krupansky -- From: Sent: Wednesday, June 02, 2010 6:45 AM To: Subject: Some more thoughts on a classloader plug-in style architecture It occurred to me that a classloader plug-in reader for LCF would not achieve the goal of allowing a fully prebuilt LCF with connector add-ons. The reason, which should have been obvious from the beginning, is because each connector consists not only of the Java implementation, but also a UI component. The UI component will need a mechanism similar to the classloader one in order for everything to work. It is possible, I suppose, for precompiled JSP's to be class-loaded instead of uncompiled JSP's in the lcf-crawler-ui.war file. However this is going to require some care and finesse (and example build.xml files) to get it to work properly. Karl
RE: Some more thoughts on a classloader plug-in style architecture
The observation about JSPs is exactly what I was alluding to. Does the entire UI have to be converted to Velocity for this approach to work? There's an intermediate path that would involve converting only the connector portions, which might be viable. Karl -Original Message- From: ext Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: Wednesday, June 02, 2010 7:27 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Actually the problem is quite tractable. We switch the UI over to Velocity templates (like Solr's VelocityResponseWriter, for example) and embed the UI bits into plugin JAR files that can live externally. JSPs aren't really workable in this fashion. Velocity templates can be loaded from the file system, the classpath, from a String, or from thin air. For example, VelocityResponseWriter allows templates to load from the actual request (clients can send in a template as an HTTP parameter), if not found it looks on the filesystem, and if not found it looks in the classpath. Erik On Jun 2, 2010, at 6:55 AM, Jack Krupansky wrote: > Good point. LCF is a bit more complex than Solr in that sense. > > Maybe a separate class is needed that has methods to retrieve the > crawl and UI components of a connector. > > Or a small XML file with whatever info about the connector is > needed. Or maybe it is simple enough for a properties file. > > Or maybe just a naming convention so that the name of the UI > component can be deduced given the logical name of a connector. > > -- Jack Krupansky > > -- > From: > Sent: Wednesday, June 02, 2010 6:45 AM > To: > Subject: Some more thoughts on a classloader plug-in style > architecture > >> It occurred to me that a classloader plug-in reader for LCF would >> not achieve the goal of allowing a fully prebuilt LCF with >> connector add-ons. The reason, which should have been obvious from >> the beginning, is because each connector consists not only of the >> Java implementation, but also a UI component. The UI component >> will need a mechanism similar to the classloader one in order for >> everything to work. >> >> It is possible, I suppose, for precompiled JSP's to be class-loaded >> instead of uncompiled JSP's in the lcf-crawler-ui.war file. >> However this is going to require some care and finesse (and example >> build.xml files) to get it to work properly. >> >> Karl >> >>
RE: Some more thoughts on a classloader plug-in style architecture
A naming convention is what is used now. This is described in the documentation. The problem is complicated because a precompiled JSP class will not fly - the package name is going to be different from application server to application server. So, either: - we abandon the thought of classloader-enabled plugin architecture, or - we recast the connector interface so that it uses something other than JSP for the plug-in contributions to the UI The latter would consist of some method that would be able to write to the response object output stream, and would have access to variables and multipart post data, but would otherwise be just a standard method. That is, there would be no JSP help whatsoever. The other downside is that it's not clear how exactly errors would be dealt with - the current convention is to use tags, and there would be no equivalent of that. Alternatively, if someone knows of a classloader-like construct present in all application servers, which could be overridden to load additional web application components, please let me know. I doubt very much there is such a beast, but you learn something new every day and maybe this would be one of them. Karl -Original Message- From: ext Jack Krupansky [mailto:jack.krupan...@lucidimagination.com] Sent: Wednesday, June 02, 2010 6:56 AM To: connectors-dev@incubator.apache.org Subject: Re: Some more thoughts on a classloader plug-in style architecture Good point. LCF is a bit more complex than Solr in that sense. Maybe a separate class is needed that has methods to retrieve the crawl and UI components of a connector. Or a small XML file with whatever info about the connector is needed. Or maybe it is simple enough for a properties file. Or maybe just a naming convention so that the name of the UI component can be deduced given the logical name of a connector. -- Jack Krupansky -- From: Sent: Wednesday, June 02, 2010 6:45 AM To: Subject: Some more thoughts on a classloader plug-in style architecture > It occurred to me that a classloader plug-in reader for LCF would not > achieve the goal of allowing a fully prebuilt LCF with connector add-ons. > The reason, which should have been obvious from the beginning, is because > each connector consists not only of the Java implementation, but also a UI > component. The UI component will need a mechanism similar to the > classloader one in order for everything to work. > > It is possible, I suppose, for precompiled JSP's to be class-loaded > instead of uncompiled JSP's in the lcf-crawler-ui.war file. However this > is going to require some care and finesse (and example build.xml files) to > get it to work properly. > > Karl > > >
Re: Some more thoughts on a classloader plug-in style architecture
Actually the problem is quite tractable. We switch the UI over to Velocity templates (like Solr's VelocityResponseWriter, for example) and embed the UI bits into plugin JAR files that can live externally. JSPs aren't really workable in this fashion. Velocity templates can be loaded from the file system, the classpath, from a String, or from thin air. For example, VelocityResponseWriter allows templates to load from the actual request (clients can send in a template as an HTTP parameter), if not found it looks on the filesystem, and if not found it looks in the classpath. Erik On Jun 2, 2010, at 6:55 AM, Jack Krupansky wrote: Good point. LCF is a bit more complex than Solr in that sense. Maybe a separate class is needed that has methods to retrieve the crawl and UI components of a connector. Or a small XML file with whatever info about the connector is needed. Or maybe it is simple enough for a properties file. Or maybe just a naming convention so that the name of the UI component can be deduced given the logical name of a connector. -- Jack Krupansky -- From: Sent: Wednesday, June 02, 2010 6:45 AM To: Subject: Some more thoughts on a classloader plug-in style architecture It occurred to me that a classloader plug-in reader for LCF would not achieve the goal of allowing a fully prebuilt LCF with connector add-ons. The reason, which should have been obvious from the beginning, is because each connector consists not only of the Java implementation, but also a UI component. The UI component will need a mechanism similar to the classloader one in order for everything to work. It is possible, I suppose, for precompiled JSP's to be class-loaded instead of uncompiled JSP's in the lcf-crawler-ui.war file. However this is going to require some care and finesse (and example build.xml files) to get it to work properly. Karl
Re: Some more thoughts on a classloader plug-in style architecture
Good point. LCF is a bit more complex than Solr in that sense. Maybe a separate class is needed that has methods to retrieve the crawl and UI components of a connector. Or a small XML file with whatever info about the connector is needed. Or maybe it is simple enough for a properties file. Or maybe just a naming convention so that the name of the UI component can be deduced given the logical name of a connector. -- Jack Krupansky -- From: Sent: Wednesday, June 02, 2010 6:45 AM To: Subject: Some more thoughts on a classloader plug-in style architecture It occurred to me that a classloader plug-in reader for LCF would not achieve the goal of allowing a fully prebuilt LCF with connector add-ons. The reason, which should have been obvious from the beginning, is because each connector consists not only of the Java implementation, but also a UI component. The UI component will need a mechanism similar to the classloader one in order for everything to work. It is possible, I suppose, for precompiled JSP's to be class-loaded instead of uncompiled JSP's in the lcf-crawler-ui.war file. However this is going to require some care and finesse (and example build.xml files) to get it to work properly. Karl
Some more thoughts on a classloader plug-in style architecture
It occurred to me that a classloader plug-in reader for LCF would not achieve the goal of allowing a fully prebuilt LCF with connector add-ons. The reason, which should have been obvious from the beginning, is because each connector consists not only of the Java implementation, but also a UI component. The UI component will need a mechanism similar to the classloader one in order for everything to work. It is possible, I suppose, for precompiled JSP's to be class-loaded instead of uncompiled JSP's in the lcf-crawler-ui.war file. However this is going to require some care and finesse (and example build.xml files) to get it to work properly. Karl