Connector architecture question and suggestion
Hi, I am having a look at the connectors. At the moment to my opinion the classes for (all) connectors are to big. This is partly due to the way the interfaces are structured and partly due to the implementation of html in java. For example the RssConnector now has almost 6000 lines of code and the JdbcConnector has almost 2000 lines of code. To my opinion this can be improved by making separate components for presenting and configuring the connectors in the crawler-ui web project and for the part needed by the runner. Abstracting the html from the actual classes can help a lot as well. Maybe some utility methods to make creating these html pages easier is nice as well. I am willing to investigate this path further, but I'd like to have ideas of other developers. It would be nice to know if others feel like this can be improved as well. It might be interesting to look at a technique like wicket for the ui part. Than you can package the html code together with the java code in one jar. No difficult repackaging is required and you can still create nice interfaces. I also read that others want to have a look at something like velocity, of course that can be a valid option as well. So what is your opinions about it? regards Jettro Coenradie http://www.gridshore.nl
Suggestion for a Hippo repository connector
Hi, We have a basic hippo repository connection that we could make available for the LCF project. Are there people that would like to have such a connector? regards -- Jettro Coenradie http://www.gridshore.nl
improving the build
Hi, I think the current download is pretty big. Is there is good reason that we do not use something like maven. Or at least, if you do not like maven, ivy to share dependencies between projects. Also enforces you to use libraries that are generally available. I would also love to have the (unit)tests closer to the actual code, hard to locate the tests at the moment Would like to hear the thoughts of the developers regards -- Jettro Coenradie http://www.gridshore.nl
Re: Suggestion for a Hippo repository connector
I will do that, need to check for the libraries that Hippo needs, will create a report and come back with that. On Mon, Aug 16, 2010 at 2:52 PM, wrote: > Connector contributions are always welcome, even if others are not using it > as of yet. Can you create a ticket which describes the connector and its > environment (including the security model the repository uses, and what > authority connector is appropriate to use with it), and attach your > connector as a patch? > > Thanks! > Karl > > > -Original Message- > From: jettro.coenra...@gmail.com [mailto:jettro.coenra...@gmail.com] On > Behalf Of ext Jettro Coenradie > Sent: Monday, August 16, 2010 8:07 AM > To: connectors-dev@incubator.apache.org > Subject: Suggestion for a Hippo repository connector > > Hi, > We have a basic hippo repository connection that we could make available > for > the LCF project. Are there people that would like to have such a connector? > > regards > > -- > Jettro Coenradie > http://www.gridshore.nl > -- Jettro Coenradie http://www.gridshore.nl
Re: improving the build
We could use something like profiles in maven. That way you can easily configure which projects to compile and which not. That would include tests. I will have a look at it and come up with a proposal. On Mon, Aug 16, 2010 at 2:49 PM, wrote: > Re maven: There is a wiki page describing the Maven dependencies; somebody > needs to tackle this. If you want to volunteer, we'd love to hear your > proposal. However, please remember that you really must be sure to retain > the connector conditional compilation structure as is currently in place, > for license reasons. > > Re unit tests: The Junit test code was actually placed carefully based on > the above considerations. In other words, you can't run a test that > requires connectors x,y,z unless those connectors have actually been built. > Similarly, you may think in terms of testing a specific connector by > including tests for that connector, but those tests cannot use any OTHER > connectors or you will break the build, which is why any tests that use > multiple connectors must be at the module level. > > Karl > > -Original Message- > From: jettro.coenra...@gmail.com [mailto:jettro.coenra...@gmail.com] On > Behalf Of ext Jettro Coenradie > Sent: Monday, August 16, 2010 8:21 AM > To: connectors-dev@incubator.apache.org > Subject: improving the build > > Hi, > I think the current download is pretty big. Is there is good reason that we > do not use something like maven. Or at least, if you do not like maven, ivy > to share dependencies between projects. Also enforces you to use libraries > that are generally available. > > I would also love to have the (unit)tests closer to the actual code, hard > to > locate the tests at the moment > > Would like to hear the thoughts of the developers > > regards > > -- > Jettro Coenradie > http://www.gridshore.nl > -- Jettro Coenradie http://www.gridshore.nl
Re: Connector architecture question and suggestion
Oke, I will come up with a proposal for breaking up the components in a way that still enables us to easily keep the current adapters in their own structure. As for the UI testen, it is always a beast. We do have some experience with tools like Webdriver and others. Will have a look at that as well. On Mon, Aug 16, 2010 at 2:41 PM, wrote: > I think that providing tools/help for implementing the UI pieces of > connectors is a perfectly reasonable thing to do. However, I strongly > believe that the UI components should remain described as part of the > connector interfaces. Breaking their implementations out within individual > connectors is also reasonable, but unless there is a compelling reason to > refactor all the individual connectors in that way, I would hold off that > project until there are better unit tests for the connector UI components. > If you are willing to contribute such tests, I think going that way would be > worth consideration. > > I'd like to see a more detailed proposal before I comment further. > > Thanks, > Karl > > -Original Message- > From: jettro.coenra...@gmail.com [mailto:jettro.coenra...@gmail.com] On > Behalf Of ext Jettro Coenradie > Sent: Monday, August 16, 2010 7:37 AM > To: connectors-dev@incubator.apache.org > Subject: Connector architecture question and suggestion > > Hi, > I am having a look at the connectors. At the moment to my opinion the > classes for (all) connectors are to big. This is partly due to the way the > interfaces are structured and partly due to the implementation of html in > java. For example the RssConnector now has almost 6000 lines of code and > the > JdbcConnector has almost 2000 lines of code. To my opinion this can be > improved by making separate components for presenting and configuring the > connectors in the crawler-ui web project and for the part needed by the > runner. Abstracting the html from the actual classes can help a lot as > well. > Maybe some utility methods to make creating these html pages easier is nice > as well. > > I am willing to investigate this path further, but I'd like to have ideas > of > other developers. It would be nice to know if others feel like this can be > improved as well. > > It might be interesting to look at a technique like wicket for the ui part. > Than you can package the html code together with the java code in one jar. > No difficult repackaging is required and you can still create nice > interfaces. I also read that others want to have a look at something like > velocity, of course that can be a valid option as well. > > So what is your opinions about it? > > regards > > Jettro Coenradie > http://www.gridshore.nl > -- Jettro Coenradie http://www.gridshore.nl
Re: Name change official
Personally I would go for it all the way. You will drag the weird package name and some class names along for all the years and releases to come. - Jettro Coenradie On Fri, Aug 20, 2010 at 3:01 PM, wrote: > There are a number of people in the field that I am aware of that have > already written their own connectors, and have working installations. I am > not trying to belittle the benefits, but I think the bar is a bit higher > that you think here. This is, after all, August, and things have been well > underway for seven months now. > > The release question has to do mostly with documentation at this point. I > believe there was one requested feature that may be done (notification to > connectors of job start and end), and one controversial feature (writing a > postgresql installer) that I didn't sign up for. Everything else was > documentation and infrastructure (e.g. nightly builds, official releases, > etc.) > > Karl > > -Original Message- > From: ext Grant Ingersoll [mailto:gsing...@apache.org] > Sent: Friday, August 20, 2010 8:51 AM > To: connectors-dev@incubator.apache.org > Subject: Re: Name change official > > > On Aug 19, 2010, at 7:44 PM, < > karl.wri...@nokia.com> wrote: > > > Changing all the java class and package names would, at this point, > seriously inconvenience our broader community. That's why I proposed it the > way I did. Unless you can think of a compelling reason to do it that way, I > think we're stuck with those as-is. > > Since we haven't done a release, I think we should do it before the > release. People on trunk, pre-release are subject to change. Besides, w/ a > modern IDE, something like this is a trivial find and replace. > > Speaking of releases... Where do people think we are in terms of putting > out something like a 0.1? > > -Grant > -- Jettro Coenradie http://www.gridshore.nl
Moving test support classes
Hi, I see a lot of classes that contain the comment : "This class is used during testing." Is it a good idea to move this classes to a separate package or even better, closer to the tests? That way they do not pollute the normal code base. a few examples: org.apache.lcf.crawler.AbortJob org.apache.lcf.crawler.AddScheduledTime org.apache.lcf.crawler.ChangeJobDocSpec etc regards -- Jettro Coenradie http://www.gridshore.nl
Re: Need an opinion, on whether to change package or not
+1 for a complete change On Mon, Aug 23, 2010 at 6:34 AM, Mark Miller wrote: > +1 to renaming the package - nows the time. > > - Mark > > http://www.lucidimagination.com (mobile) > > On Aug 22, 2010, at 8:01 PM, "Jack Krupansky" < > jack.krupan...@lucidimagination.com> wrote: > > > +1 > > > > -- Jack Krupansky > > > > -- > > From: "Karl Wright" > > Sent: Sunday, August 22, 2010 1:49 PM > > To: "connectors-dev" > > Subject: Need an opinion, on whether to change package or not > > > >> Consider this an official request for a vote. > >> > >> +1 indicates you think we should change the following in the source > code, as > >> soon as is practical: > >> > >> org.apache.lcf.xxx -> org.apache.acf.xxx > >> All classes LCF.java and LCFException.java should change to ACF.java and > >> ACFException.java > >> > >> Bear in mind that users of ACF/LCF who currently have existing database > >> instances will need to reinitialize those instances if we do this > change. > >> This is because the class names of connectors are stored in the database > >> when the connector is registered. > >> > >> (FWIW, my vote on this is -1. It doesn't seem worth the disruption. > But I > >> will of course abide by the consensus.) > >> > >> Vote will be considered closed by Wednesday evening, so vote early (and > >> often. ;-)) > >> Karl > -- Jettro Coenradie http://www.gridshore.nl
Re: Need an opinion, on whether to change package or not
If we are changing stuff can we also use more descriptive names. Not Use LCF 4 to 5 times in a different Package. Use something like ACFAgent and ACFCrawler On Mon, Aug 23, 2010 at 10:33 AM, Jettro Coenradie < jettro.coenra...@gridshore.nl> wrote: > +1 for a complete change > > > On Mon, Aug 23, 2010 at 6:34 AM, Mark Miller wrote: > >> +1 to renaming the package - nows the time. >> >> - Mark >> >> http://www.lucidimagination.com (mobile) >> >> On Aug 22, 2010, at 8:01 PM, "Jack Krupansky" < >> jack.krupan...@lucidimagination.com> wrote: >> >> > +1 >> > >> > -- Jack Krupansky >> > >> > -- >> > From: "Karl Wright" >> > Sent: Sunday, August 22, 2010 1:49 PM >> > To: "connectors-dev" >> > Subject: Need an opinion, on whether to change package or not >> > >> >> Consider this an official request for a vote. >> >> >> >> +1 indicates you think we should change the following in the source >> code, as >> >> soon as is practical: >> >> >> >> org.apache.lcf.xxx -> org.apache.acf.xxx >> >> All classes LCF.java and LCFException.java should change to ACF.java >> and >> >> ACFException.java >> >> >> >> Bear in mind that users of ACF/LCF who currently have existing database >> >> instances will need to reinitialize those instances if we do this >> change. >> >> This is because the class names of connectors are stored in the >> database >> >> when the connector is registered. >> >> >> >> (FWIW, my vote on this is -1. It doesn't seem worth the disruption. >> But I >> >> will of course abide by the consensus.) >> >> >> >> Vote will be considered closed by Wednesday evening, so vote early (and >> >> often. ;-)) >> >> Karl >> > > > > -- > Jettro Coenradie > http://www.gridshore.nl > -- Jettro Coenradie http://www.gridshore.nl
Re: Need an opinion, on whether to change package or not
I can understand that it is harder to do. Therefore it is better notto do it right now. I do not agree with you that it is easier to move files from one package to another. The fact that these classes have different impact should make you think before moving the classes. I would like to discuss on some of these design/code issues more as well. What is the best way to do this? Ask a question per topic to share opinions? thanks Jettro On Mon, Aug 23, 2010 at 10:54 AM, wrote: > Unfortunately that is way harder to do using the python scripts I have > developed for this purpose. Also, the reason the LCF root class appears in > different packages has to do with the relative ease that grants to moving > classes between the various acf jars. So I'd consider this proposed change > to be controversial, and I don't think we should layer it in without > separate consideration. > > Karl > > > From: ext Simon Willnauer [simon.willna...@googlemail.com] > Sent: Monday, August 23, 2010 4:40 AM > To: connectors-dev@incubator.apache.org > Subject: Re: Need an opinion, on whether to change package or not > > On Mon, Aug 23, 2010 at 10:35 AM, Jettro Coenradie > wrote: > > If we are changing stuff can we also use more descriptive names. Not Use > LCF > > 4 to 5 times in a different Package. Use something like ACFAgent and > > ACFCrawler > +1 for that too! > > simon > > > > On Mon, Aug 23, 2010 at 10:33 AM, Jettro Coenradie < > > jettro.coenra...@gridshore.nl> wrote: > > > >> +1 for a complete change > >> > >> > >> On Mon, Aug 23, 2010 at 6:34 AM, Mark Miller >wrote: > >> > >>> +1 to renaming the package - nows the time. > >>> > >>> - Mark > >>> > >>> http://www.lucidimagination.com (mobile) > >>> > >>> On Aug 22, 2010, at 8:01 PM, "Jack Krupansky" < > >>> jack.krupan...@lucidimagination.com> wrote: > >>> > >>> > +1 > >>> > > >>> > -- Jack Krupansky > >>> > > >>> > -- > >>> > From: "Karl Wright" > >>> > Sent: Sunday, August 22, 2010 1:49 PM > >>> > To: "connectors-dev" > >>> > Subject: Need an opinion, on whether to change package or not > >>> > > >>> >> Consider this an official request for a vote. > >>> >> > >>> >> +1 indicates you think we should change the following in the source > >>> code, as > >>> >> soon as is practical: > >>> >> > >>> >> org.apache.lcf.xxx -> org.apache.acf.xxx > >>> >> All classes LCF.java and LCFException.java should change to ACF.java > >>> and > >>> >> ACFException.java > >>> >> > >>> >> Bear in mind that users of ACF/LCF who currently have existing > database > >>> >> instances will need to reinitialize those instances if we do this > >>> change. > >>> >> This is because the class names of connectors are stored in the > >>> database > >>> >> when the connector is registered. > >>> >> > >>> >> (FWIW, my vote on this is -1. It doesn't seem worth the disruption. > >>> But I > >>> >> will of course abide by the consensus.) > >>> >> > >>> >> Vote will be considered closed by Wednesday evening, so vote early > (and > >>> >> often. ;-)) > >>> >> Karl > >>> > >> > >> > >> > >> -- > >> Jettro Coenradie > >> http://www.gridshore.nl > >> > > > > > > > > -- > > Jettro Coenradie > > http://www.gridshore.nl > > > -- Jettro Coenradie http://www.gridshore.nl
Re: Need an opinion, on whether to change package or not
Cool, thanks I don't want to be a pain, but trying to help improve the end result. And of course I understand the project has a history and I know not everybody thinks like me :-) Jettro On Mon, Aug 23, 2010 at 11:46 AM, wrote: > In any open-source project there is expected to be some differences in > individual coding styles. There is often also incomplete understanding of > the reasoning behind the multitude of architectural decisions made during > development, or the history of the project. It is thus important to be > pragmatic, and therefore each issue or question is basically its own topic, > evaluated on its own merits. > > Probably the best way to deal with each *individual* concern or question is > to open a jira ticket expressing that concern. Discussion should then be > done within the context of that ticket. There is no guarantee, of course, > that the ticket will be acted upon, but at least it will be discussed. > > Karl > > > From: jettro.coenra...@gmail.com [jettro.coenra...@gmail.com] On Behalf Of > ext Jettro Coenradie [jettro.coenra...@gridshore.nl] > Sent: Monday, August 23, 2010 5:17 AM > To: connectors-dev@incubator.apache.org > Subject: Re: Need an opinion, on whether to change package or not > > I can understand that it is harder to do. Therefore it is better notto do > it > right now. I do not agree with you that it is easier to move files from one > package to another. The fact that these classes have different impact > should > make you think before moving the classes. I would like to discuss on some > of > these design/code issues more as well. What is the best way to do this? Ask > a question per topic to share opinions? > > thanks Jettro > > On Mon, Aug 23, 2010 at 10:54 AM, wrote: > > > Unfortunately that is way harder to do using the python scripts I have > > developed for this purpose. Also, the reason the LCF root class appears > in > > different packages has to do with the relative ease that grants to moving > > classes between the various acf jars. So I'd consider this proposed > change > > to be controversial, and I don't think we should layer it in without > > separate consideration. > > > > Karl > > > > > > From: ext Simon Willnauer [simon.willna...@googlemail.com] > > Sent: Monday, August 23, 2010 4:40 AM > > To: connectors-dev@incubator.apache.org > > Subject: Re: Need an opinion, on whether to change package or not > > > > On Mon, Aug 23, 2010 at 10:35 AM, Jettro Coenradie > > wrote: > > > If we are changing stuff can we also use more descriptive names. Not > Use > > LCF > > > 4 to 5 times in a different Package. Use something like ACFAgent and > > > ACFCrawler > > +1 for that too! > > > > simon > > > > > > On Mon, Aug 23, 2010 at 10:33 AM, Jettro Coenradie < > > > jettro.coenra...@gridshore.nl> wrote: > > > > > >> +1 for a complete change > > >> > > >> > > >> On Mon, Aug 23, 2010 at 6:34 AM, Mark Miller > >wrote: > > >> > > >>> +1 to renaming the package - nows the time. > > >>> > > >>> - Mark > > >>> > > >>> http://www.lucidimagination.com (mobile) > > >>> > > >>> On Aug 22, 2010, at 8:01 PM, "Jack Krupansky" < > > >>> jack.krupan...@lucidimagination.com> wrote: > > >>> > > >>> > +1 > > >>> > > > >>> > -- Jack Krupansky > > >>> > > > >>> > -- > > >>> > From: "Karl Wright" > > >>> > Sent: Sunday, August 22, 2010 1:49 PM > > >>> > To: "connectors-dev" > > >>> > Subject: Need an opinion, on whether to change package or not > > >>> > > > >>> >> Consider this an official request for a vote. > > >>> >> > > >>> >> +1 indicates you think we should change the following in the > source > > >>> code, as > > >>> >> soon as is practical: > > >>> >> > > >>> >> org.apache.lcf.xxx -> org.apache.acf.xxx > > >>> >> All classes LCF.java and LCFException.java should change to > ACF.java > > >>> and > > >>> >> ACFException.java > > >>> >> > > >>> >> Bear in mind that users of ACF/LCF who currently have existing > > database > > >>> >> instances will need to reinitialize those instances if we do this > > >>> change. > > >>> >> This is because the class names of connectors are stored in the > > >>> database > > >>> >> when the connector is registered. > > >>> >> > > >>> >> (FWIW, my vote on this is -1. It doesn't seem worth the > disruption. > > >>> But I > > >>> >> will of course abide by the consensus.) > > >>> >> > > >>> >> Vote will be considered closed by Wednesday evening, so vote early > > (and > > >>> >> often. ;-)) > > >>> >> Karl > > >>> > > >> > > >> > > >> > > >> -- > > >> Jettro Coenradie > > >> http://www.gridshore.nl > > >> > > > > > > > > > > > > -- > > > Jettro Coenradie > > > http://www.gridshore.nl > > > > > > > > > -- > Jettro Coenradie > http://www.gridshore.nl > -- Jettro Coenradie http://www.gridshore.nl
Question about the json library
I am looking at the classes that come with the current trunk checkout and I see that a custom jar of json is created. Can someone explain why this is? Could we also take one from a maven repository? thanks, Jettro -- Jettro Coenradie http://www.gridshore.nl
Re: Question about the json library
I know maven is another issue, but it is nice if the version is available through a maven repository. Than other build tools can find it as well. It is available for download through: http://mvnrepository.com/artifact/org.json/json or from the central maven repo: http://repo2.maven.org/maven2/org/json/json/20090211/json-20090211.jar Jettro On Mon, Aug 23, 2010 at 4:16 PM, wrote: > The sources were downloaded from www.json.org, and are licensed > accordingly. There is no build available from www.json.org. If you know > of a prebuilt version of these sources, by all means point us at it. > > Mavenization is a different issue, and will have to be done independently. > > Karl > > -Original Message- > From: jettro.coenra...@gmail.com [mailto:jettro.coenra...@gmail.com] On > Behalf Of ext Jettro Coenradie > Sent: Monday, August 23, 2010 10:04 AM > To: connectors-dev@incubator.apache.org > Subject: Question about the json library > > I am looking at the classes that come with the current trunk checkout and I > see that a custom jar of json is created. Can someone explain why this is? > Could we also take one from a maven repository? > > thanks, > Jettro > > -- > Jettro Coenradie > http://www.gridshore.nl > -- Jettro Coenradie http://www.gridshore.nl
Re: improving the build
I am looking at the current project structure. If we want to make another build tool available I think we need to change the directory structure. I tried to place a suggestion in an image. Can you please have a look at it. If we agree that this is a good way to go, than I will continue to work on a patch. Which might be a bit hard with all these changing directories, but I'll do my best to at least get an idea whether it would be working. So I have three questions: - Do you want to move to maven or put maven next to ant? - Do you prefer another build mechanism [ant with ivy, gradle, maven3] - Do you have an idea about the amount of scripts that need to be changed if we change the project structure The image of a possible project layout (that is based on the maven standards) is attached to the mail Jettro On Mon, Aug 16, 2010 at 3:12 PM, Jettro Coenradie < jettro.coenra...@gridshore.nl> wrote: > We could use something like profiles in maven. That way you can easily > configure which projects to compile and which not. That would include tests. > > I will have a look at it and come up with a proposal. > > > On Mon, Aug 16, 2010 at 2:49 PM, wrote: > >> Re maven: There is a wiki page describing the Maven dependencies; somebody >> needs to tackle this. If you want to volunteer, we'd love to hear your >> proposal. However, please remember that you really must be sure to retain >> the connector conditional compilation structure as is currently in place, >> for license reasons. >> >> Re unit tests: The Junit test code was actually placed carefully based on >> the above considerations. In other words, you can't run a test that >> requires connectors x,y,z unless those connectors have actually been built. >> Similarly, you may think in terms of testing a specific connector by >> including tests for that connector, but those tests cannot use any OTHER >> connectors or you will break the build, which is why any tests that use >> multiple connectors must be at the module level. >> >> Karl >> >> -Original Message- >> From: jettro.coenra...@gmail.com [mailto:jettro.coenra...@gmail.com] On >> Behalf Of ext Jettro Coenradie >> Sent: Monday, August 16, 2010 8:21 AM >> To: connectors-dev@incubator.apache.org >> Subject: improving the build >> >> Hi, >> I think the current download is pretty big. Is there is good reason that >> we >> do not use something like maven. Or at least, if you do not like maven, >> ivy >> to share dependencies between projects. Also enforces you to use libraries >> that are generally available. >> >> I would also love to have the (unit)tests closer to the actual code, hard >> to >> locate the tests at the moment >> >> Would like to hear the thoughts of the developers >> >> regards >> >> -- >> Jettro Coenradie >> http://www.gridshore.nl >> > > > > -- > Jettro Coenradie > http://www.gridshore.nl > -- Jettro Coenradie http://www.gridshore.nl
Re: improving the build
That is a good idea, maven can call ant to execute tasks. The jars are available in the maven repository and should therefore not be to hard to make available to the ant build. Would be nice to have an idea of the amount of scripts that we need to alter to make this happen. I also see a lot of shell scripts that might need attention if we change this. - Jettro On Mon, Aug 23, 2010 at 4:52 PM, wrote: > Re: build preferences > > Continuing to have an ant build is actually pretty important for some modes > of delivery. I'm specifically thinking of debian and Ubuntu packaging here. > Maven does not work well with these packaging schemes because it's too > all-encompassing. We therefore need a way of doing builds locally, without > pulling things down from a mirror. > > My original thought was that we'd have multiple layers - ant being the > most basic, with a maven wrapper available to pull down what the ant build > needed, and have the maven build call ant underneath. I don't know how > realistic that is, but it does solve all the problems if it can be done that > way. > > Karl > > From: jettro.coenra...@gmail.com [mailto:jettro.coenra...@gmail.com] On > Behalf Of ext Jettro Coenradie > Sent: Monday, August 23, 2010 10:43 AM > To: connectors-dev@incubator.apache.org > Subject: Re: improving the build > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. > If we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed > if we change the project structure > > The image of a possible project layout (that is based on the maven > standards) is attached to the mail > > Jettro > On Mon, Aug 16, 2010 at 3:12 PM, Jettro Coenradie < > jettro.coenra...@gridshore.nl<mailto:jettro.coenra...@gridshore.nl>> > wrote: > We could use something like profiles in maven. That way you can easily > configure which projects to compile and which not. That would include tests. > > I will have a look at it and come up with a proposal. > > On Mon, Aug 16, 2010 at 2:49 PM, karl.wri...@nokia.com>> wrote: > Re maven: There is a wiki page describing the Maven dependencies; somebody > needs to tackle this. If you want to volunteer, we'd love to hear your > proposal. However, please remember that you really must be sure to retain > the connector conditional compilation structure as is currently in place, > for license reasons. > > Re unit tests: The Junit test code was actually placed carefully based on > the above considerations. In other words, you can't run a test that > requires connectors x,y,z unless those connectors have actually been built. > Similarly, you may think in terms of testing a specific connector by > including tests for that connector, but those tests cannot use any OTHER > connectors or you will break the build, which is why any tests that use > multiple connectors must be at the module level. > > Karl > > -Original Message- > From: jettro.coenra...@gmail.com<mailto:jettro.coenra...@gmail.com> > [mailto:jettro.coenra...@gmail.com<mailto:jettro.coenra...@gmail.com>] On > Behalf Of ext Jettro Coenradie > Sent: Monday, August 16, 2010 8:21 AM > To: connectors-dev@incubator.apache.org connectors-dev@incubator.apache.org> > Subject: improving the build > > Hi, > I think the current download is pretty big. Is there is good reason that we > do not use something like maven. Or at least, if you do not like maven, ivy > to share dependencies between projects. Also enforces you to use libraries > that are generally available. > > I would also love to have the (unit)tests closer to the actual code, hard > to > locate the tests at the moment > > Would like to hear the thoughts of the developers > > regards > > -- > Jettro Coenradie > http://www.gridshore.nl > > > > -- > Jettro Coenradie > http://www.gridshore.nl > > > > -- > Jettro Coenradie > http://www.gridshore.nl > -- Jettro Coenradie http://www.gridshore.nl
Re: [VOTE] Pick your preferred name
Apache Yukon Apache Macon Apache Manifold On Wed, Sep 1, 2010 at 9:42 AM, Simon Willnauer < simon.willna...@googlemail.com> wrote: > Apache Manifold > Apache Connectors Framework > Apache Omni > > On Wed, Sep 1, 2010 at 2:21 AM, Karl Wright wrote: > > I think this vote will conclusively demonstrate the community position to > > the solons at incubator. They can choose to ignore the vote, of course, > but > > they won't be able to say we didn't consider alternatives. ;-) > > > > Karl > > > > On Tue, Aug 31, 2010 at 8:16 PM, Mark Miller > wrote: > > > >> I think its pretty clear that unless the lords of the incubator *force* > >> us to do otherwise, we should just leave it as Apache Connectors > >> Framework. The vote is going to end up reflecting that is my bet anyway. > >> I havn't seen anyone that's actually involved (if you can say anyone > >> beyond karl is really involved anyway) is really dying for a change. > >> > >> - Mark > >> > >> On 8/31/10 8:07 PM, Robert Muir wrote: > >> > you can ignore my vote really :) > >> > > >> > but i was thinking from a practical perspective, this one involves the > >> least > >> > code changes. > >> > > >> > plus, with existing projects named things like "apache DB" and "apache > >> http > >> > server", i don't understand the fuss. > >> > > >> > On Tue, Aug 31, 2010 at 8:03 PM, Karl Wright > wrote: > >> > > >> >> Well, you've made your feelings clear. ;-) > >> >> Karl > >> >> > >> >> On Tue, Aug 31, 2010 at 7:58 PM, Robert Muir > wrote: > >> >> > >> >>> Apache Connectors Framework > >> >>> Apache Connectors Framework > >> >>> Apache Connectors Framework > >> >>> > >> >>> (sorry) > >> >>> > >> >>> On Tue, Aug 31, 2010 at 6:44 PM, Karl Wright > >> wrote: > >> >>> > >> >>>> I know this is un-Apache-like, but please respond to the following > >> list > >> >>>> with > >> >>>> a selection, in order, of the top three names for the project > >> currently > >> >>>> known as Apache Connectors Framework. The choices > >> >>>> are: > >> >>>> > >> >>>> Apache Connectors Framework > >> >>>> Apache Acromantula > >> >>>> Apache Manifold > >> >>>> Apache ManifoldCF > >> >>>> Apache Multiplex > >> >>>> Apache Lucon > >> >>>> Apache Lukon > >> >>>> Apache Yukon > >> >>>> Apache Macon > >> >>>> Apache Omni > >> >>>> Apache Omnivore > >> >>>> Apache CMCF (yes, I just invented that one ;-) ) > >> >>>> Apache Multivore (yes, I just invented that one too. ;-) ) > >> >>>> > >> >>>> I don't think I missed any? If I did, chastise me severely please. > >> ;-) > >> >>>> > >> >>>> Karl > >> >>>> > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> Robert Muir > >> >>> rcm...@gmail.com > >> >>> > >> >> > >> > > >> > > >> > > >> > >> > > > -- Jettro Coenradie http://www.gridshore.nl
[jira] Created: (CONNECTORS-91) Making the initialization commands more useable
Making the initialization commands more useable --- Key: CONNECTORS-91 URL: https://issues.apache.org/jira/browse/CONNECTORS-91 Project: Lucene Connector Framework Issue Type: Improvement Components: Framework core Reporter: Jettro Coenradie Fix For: LCF Release 0.5 At the moment LCF comes with some classes that can be used to run command line to interact with the system. Examples are DBCreate, DBDrop and LockClean. I wanted to create a class that rebuilds my complete environment. So dropping a database, creating a database, cleaning the synch folder, registering agents, etc. Due to the structure of the classes with all the logic in the main method, I could not easily reuse these classes. In the patch I submit with issue I have refactored the current solution in a better reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-91: --- Attachment: commandsPatch.patch the patch with an improvement for the commands > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Lucene Connector Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: commandsPatch.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898857#action_12898857 ] Jettro Coenradie commented on CONNECTORS-91: If you feel this is the way to go, I will change the other classes that have a main method as well. > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Lucene Connector Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: commandsPatch.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-19) Look into converting SOLR connector to use SolrJ java library
[ https://issues.apache.org/jira/browse/CONNECTORS-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898859#action_12898859 ] Jettro Coenradie commented on CONNECTORS-19: We have a working solr connector that makes use of solr. This might be a good start. I might need to spend some time to make it run in the lcf build. We have a maven build to package it at the moment. If you are interested, let me know. Than I will spend the time on a patch. > Look into converting SOLR connector to use SolrJ java library > - > > Key: CONNECTORS-19 > URL: https://issues.apache.org/jira/browse/CONNECTORS-19 > Project: Lucene Connector Framework > Issue Type: Improvement > Components: Lucene/SOLR connector >Reporter: Karl Wright >Priority: Minor > > The SOLR connector currently uses its own multipart post code. It might be a > good idea to convert it to use the SolrJ client api jar instead. This would > require license confirmation, plus research to make sure there are no jar > conflicts as a result, with any other connector. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CONNECTORS-19) Look into converting SOLR connector to use SolrJ java library
[ https://issues.apache.org/jira/browse/CONNECTORS-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898859#action_12898859 ] Jettro Coenradie edited comment on CONNECTORS-19 at 8/16/10 7:45 AM: - We have a working solr connector that makes use of solrj. This might be a good start. I might need to spend some time to make it run in the lcf build. We have a maven build to package it at the moment. If you are interested, let me know. Than I will spend the time on a patch. was (Author: jettroc): We have a working solr connector that makes use of solr. This might be a good start. I might need to spend some time to make it run in the lcf build. We have a maven build to package it at the moment. If you are interested, let me know. Than I will spend the time on a patch. > Look into converting SOLR connector to use SolrJ java library > - > > Key: CONNECTORS-19 > URL: https://issues.apache.org/jira/browse/CONNECTORS-19 > Project: Lucene Connector Framework > Issue Type: Improvement > Components: Lucene/SOLR connector >Reporter: Karl Wright >Priority: Minor > > The SOLR connector currently uses its own multipart post code. It might be a > good idea to convert it to use the SolrJ client api jar instead. This would > require license confirmation, plus research to make sure there are no jar > conflicts as a result, with any other connector. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898925#action_12898925 ] Jettro Coenradie commented on CONNECTORS-91: There should be no subtleties, I mainly moved code from the main method into a new method and indeed a bit of class-inheritance. > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Lucene Connector Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: commandsPatch.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-19) Look into converting SOLR connector to use SolrJ java library
[ https://issues.apache.org/jira/browse/CONNECTORS-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898928#action_12898928 ] Jettro Coenradie commented on CONNECTORS-19: I will have a good look at the dependencies and the functionality. If satisfied, I will supply a patch that other can check as well. > Look into converting SOLR connector to use SolrJ java library > - > > Key: CONNECTORS-19 > URL: https://issues.apache.org/jira/browse/CONNECTORS-19 > Project: Lucene Connector Framework > Issue Type: Improvement > Components: Lucene/SOLR connector >Reporter: Karl Wright >Priority: Minor > > The SOLR connector currently uses its own multipart post code. It might be a > good idea to convert it to use the SolrJ client api jar instead. This would > require license confirmation, plus research to make sure there are no jar > conflicts as a result, with any other connector. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900844#action_12900844 ] Jettro Coenradie commented on CONNECTORS-91: I am going though the classes, but they are not all easy to do. It actually is a lot of work. Can we split the work in the most important classes first and maybe later on the. You talk about the actual command classes, can you provide a list with the classes you mean? They would help me a lot. Also for the patch it is easier to know if the package is actually going to be changed now that the name of the project is changed > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: commandsPatch.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-91: --- Attachment: changesToCommandClasses.patch I have altered all the command classes as written on the wiki page. > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: changesToCommandClasses.patch, commandsPatch.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-91: --- Status: Open (was: Patch Available) sorry, pushed the wrong button I guess > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: changesToCommandClasses.patch, commandsPatch.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-91: --- Status: Patch Available (was: Open) Some strange things are happening, not sure what went wrong. I did do an svn up, I am sure of that. Nevertheless, I think I have it working now. You might need to change the depth of which to apply the patch. I recreated the patch with intellij and it uses one folder of my own. The following command strips of this folder patch -p1 -i ~/change_commands.patch I tried it on a clean checkout of the project locally and it seems to work Hope it works now, sorry I did not try it myself before > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: changesToCommandClasses.patch, commandsPatch.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-91: --- Attachment: change_commands.patch Some strange things are happening, not sure what went wrong. I did do an svn up, I am sure of that. Nevertheless, I think I have it working now. You might need to change the depth of which to apply the patch. I recreated the patch with intellij and it uses one folder of my own. The following command strips of this folder patch -p1 -i ~/change_commands.patch I tried it on a clean checkout of the project locally and it seems to work Hope it works now, sorry I did not try it myself before > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: change_commands.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-91: --- Attachment: (was: changesToCommandClasses.patch) > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: change_commands.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-91: --- Attachment: (was: commandsPatch.patch) > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: change_commands.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-91: --- Comment: was deleted (was: sorry, pushed the wrong button I guess) > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: change_commands.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-91: --- Comment: was deleted (was: Some strange things are happening, not sure what went wrong. I did do an svn up, I am sure of that. Nevertheless, I think I have it working now. You might need to change the depth of which to apply the patch. I recreated the patch with intellij and it uses one folder of my own. The following command strips of this folder patch -p1 -i ~/change_commands.patch I tried it on a clean checkout of the project locally and it seems to work Hope it works now, sorry I did not try it myself before) > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: change_commands.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901316#action_12901316 ] Jettro Coenradie commented on CONNECTORS-91: Hmm, I think the logging option is better, if people provide the right configuration you have what you need and even more. But I understand what you mean with the main method implementation. I'll add it back and provide a new patch. I also tried the sample with the new classes and it all seems to work. Is that good enough? > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: change_commands.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-91: --- Attachment: change_commands_with_system_err_println.patch added system err println lines back to the main methods > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: change_commands.patch, > change_commands_with_system_err_println.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-91) Making the initialization commands more useable
[ https://issues.apache.org/jira/browse/CONNECTORS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-91: --- Attachment: change_commands_with_system_err_println_v2.patch Sorry about the errors, I was a little bit to quick. I double checked all locations of printing the messages and the messages themselves. They should all be fine now. > Making the initialization commands more useable > --- > > Key: CONNECTORS-91 > URL: https://issues.apache.org/jira/browse/CONNECTORS-91 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > Fix For: LCF Release 0.5 > > Attachments: change_commands.patch, > change_commands_with_system_err_println.patch, > change_commands_with_system_err_println_v2.patch > > > At the moment LCF comes with some classes that can be used to run command > line to interact with the system. Examples are DBCreate, DBDrop and > LockClean. I wanted to create a class that rebuilds my complete environment. > So dropping a database, creating a database, cleaning the synch folder, > registering agents, etc. Due to the structure of the classes with all the > logic in the main method, I could not easily reuse these classes. In the > patch I submit with issue I have refactored the current solution in a better > reuseable solution that can still be called command line. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
Move from ant to maven or other build system with decent library management --- Key: CONNECTORS-92 URL: https://issues.apache.org/jira/browse/CONNECTORS-92 Project: Apache Connectors Framework Issue Type: Wish Components: Build Reporter: Jettro Coenradie Attachments: Screen shot 2010-08-23 at 16.31.07.png I am looking at the current project structure. If we want to make another build tool available I think we need to change the directory structure. I tried to place a suggestion in an image. Can you please have a look at it. If we agree that this is a good way to go, than I will continue to work on a patch. Which might be a bit hard with all these changing directories, but I'll do my best to at least get an idea whether it would be working. So I have three questions: - Do you want to move to maven or put maven next to ant? - Do you prefer another build mechanism [ant with ivy, gradle, maven3] - Do you have an idea about the amount of scripts that need to be changed if we change the project structure The image of a possible project layout (that is based on the maven standards) is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-92: --- Attachment: Screen shot 2010-08-23 at 16.31.07.png idea of the directory structure > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie > Attachments: Screen shot 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901769#action_12901769 ] Jettro Coenradie commented on CONNECTORS-92: As a response to the remark from Karl (1) Breaking up modules and putting pieces of that all over the place I do not think they are all over the place, maybe I am thinking wrong about the modules part, but for me modules is not really clear. At the moment we have documentation, modules and tests. I suggest a slightly more separated mode with: documentation, integration-tests, framework, connectors and environment. The only change is to move some stuff from modules into a new part environment en move the other parts of modules one level up. (2) Taking jetty-runner out of framework I do not think that Jetty is part of your framework, you create war files and give the option for an easy start using Jetty. But maybe I am wrong. (3) Introducing a "src" directory under each of the framework components At the moment when running ant. You get a lot of folders of which it is not always easy to understand whether they are original source folders or not. That is why maven comes with a clear separation of src, generated-source and target for other generated content. To my opinion this makes it easier to see what is under version control and what is not. Check the maven page for more explanation. http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html (4) Moving the tests so far away from the code they are related to I am not sure if I was clear enough on this. In the original code base a test folder is available next to modules. For unit tests I would keep them as close as possible to the source code. Therefore we have the src/main and src/test in the same module. The integration tests are another beast. Usually a lot of environmental setup needs to be done, they take longer, and you might want to store them in a different folder so you can run them all at once. Another option would be to add them next to the unit tests in a different folder [src/main, src/test/ and src/integration-test] or use a different naming scheme. **Test.java and **IntegrationTest.java That way you can folder them out as well and use the maven lifecycle to decide whether to run unit test or both unit and integration tests. As for the part where we need ant to make certain deliveries, we can still do that. Than you have two options, or run the ant command to make these deliverables yourself, or add them to the package lifecycle stage of maven. It is not hard to call ant from maven. You could even create the mother of all ant build file that calls maven to build the framework and the conenctors. > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build >Reporter: Jettro Coenradie > Attachments: Screen shot 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901786#action_12901786 ] Jettro Coenradie commented on CONNECTORS-92: - Compiling C with maven : I have no experience with that - Multiple jars: I know people have done it. I would not do it however. Create a pom and structure for each module. Group the module by using a parent pom and a directory structure. - I agree that each connector should be a component on its own, with its own pom. - The fact that jetty has a dependency on the framework is no problem at all. We can just create an internal dependency and release them together if necessary. What is the next best step? Come up with a directory structure that is synchronized with our ideas of the different components? I can change the image, but we can also do something on the wiki or on the description of this issue. > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie > Attachments: Screen shot 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901795#action_12901795 ] Jettro Coenradie commented on CONNECTORS-92: I'll try to come up with something tonight. Today I am at a customer so I cannot spend to much time. Web projects are no problem at all. You can even have dependencies between webproject. Althought I would try to make dependencies on jars only. I'll come up with a directory structure and the components based on the deliverables or the targets. > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build >Reporter: Jettro Coenradie > Attachments: Screen shot 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CONNECTORS-96) Replace the current connection pool by one that is more up-to-date
Replace the current connection pool by one that is more up-to-date -- Key: CONNECTORS-96 URL: https://issues.apache.org/jira/browse/CONNECTORS-96 Project: Apache Connectors Framework Issue Type: Improvement Components: Framework core Reporter: Jettro Coenradie At the moment we use a connection pool from bitmechanic. This is the least to say some what outdated. Hard to find any information about it. The only thing I could find was this: http://www.knightsofthenet.com/projects/SQLPool/ Can we replace this with an implementation of for instance dbcp http://commons.apache.org/dbcp/index.html I think there is a big risk in using this old versions that are not tested on new java systems anymore. We can be missing a lot of optimizations (jdbc4 for instance) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-96) Replace the current connection pool by one that is more up-to-date
[ https://issues.apache.org/jira/browse/CONNECTORS-96?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-96: --- Description: At the moment we use a connection pool from bitmechanic. This is the least to say some what outdated. Hard to find any information about it. The only thing I could find was this: http://www.knightsofthenet.com/projects/SQLPool/ Can we replace this with an implementation of for instance dbcp http://commons.apache.org/dbcp/index.html I think there is a big risk in using this old versions that are not tested on new java systems anymore. We can be missing a lot of optimizations (jdbc4 for instance) For the pooling behavior we can use something like commons-pool was: At the moment we use a connection pool from bitmechanic. This is the least to say some what outdated. Hard to find any information about it. The only thing I could find was this: http://www.knightsofthenet.com/projects/SQLPool/ Can we replace this with an implementation of for instance dbcp http://commons.apache.org/dbcp/index.html I think there is a big risk in using this old versions that are not tested on new java systems anymore. We can be missing a lot of optimizations (jdbc4 for instance) > Replace the current connection pool by one that is more up-to-date > -- > > Key: CONNECTORS-96 > URL: https://issues.apache.org/jira/browse/CONNECTORS-96 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > > At the moment we use a connection pool from bitmechanic. This is the least to > say some what outdated. Hard to find any information about it. The only thing > I could find was this: > http://www.knightsofthenet.com/projects/SQLPool/ > Can we replace this with an implementation of for instance dbcp > http://commons.apache.org/dbcp/index.html > I think there is a big risk in using this old versions that are not tested on > new java systems anymore. We can be missing a lot of optimizations (jdbc4 for > instance) > For the pooling behavior we can use something like commons-pool -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-96) Replace the current connection pool by one that is more up-to-date
[ https://issues.apache.org/jira/browse/CONNECTORS-96?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902762#action_12902762 ] Jettro Coenradie commented on CONNECTORS-96: This is a good example of what we could do: http://svn.apache.org/viewvc/commons/proper/dbcp/trunk/doc/ManualPoolingDataSourceExample.java?view=markup > Replace the current connection pool by one that is more up-to-date > -- > > Key: CONNECTORS-96 > URL: https://issues.apache.org/jira/browse/CONNECTORS-96 > Project: Apache Connectors Framework > Issue Type: Improvement > Components: Framework core > Reporter: Jettro Coenradie > > At the moment we use a connection pool from bitmechanic. This is the least to > say some what outdated. Hard to find any information about it. The only thing > I could find was this: > http://www.knightsofthenet.com/projects/SQLPool/ > Can we replace this with an implementation of for instance dbcp > http://commons.apache.org/dbcp/index.html > I think there is a big risk in using this old versions that are not tested on > new java systems anymore. We can be missing a lot of optimizations (jdbc4 for > instance) > For the pooling behavior we can use something like commons-pool -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-92: --- Attachment: move-to-maven-acf-framework.patch This is a first go on moving to maven. I started refactoring the framework part. This should give a good idea of what is possible. Of course we are nog there. I did not really touch any of the scripts. There is also one dependency you have to install locally. You'll see it when you try to run maven. Advice on ho to install it locally is provided when it fails. I am not sure if this patch will run, it is a big one. Hope for the best When the patch is successfully applied, go to the modules/framework folder and try: mvn clean install then step into crawler-ui and do mvn clean jetty:run-war Use you browser to go to localhost:8080 and you can see the crawler-ui web app. If you connect to an existing database, you should be able to see the configure connections lists. Not more than that, than we need to put the connectors on the classpath. I'd be happy to do something similar for the connectors. But than we must be sure that this is the way to go. It takes a lot of time. > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie > Attachments: move-to-maven-acf-framework.patch, Screen shot > 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902814#action_12902814 ] Jettro Coenradie commented on CONNECTORS-92: Sorry, but why is it easier to have one src for the complete framework. To my opinion the source code becomes a lot easier to understand if you make it explicit what the different modules are. Besides that, maven just does not work that way. It is also a lot harder when depending on these parts. The way I see it, we have already three different war files. The other jars are shared between the three war files. This way it becomes clear what deliverables there are without looking at the code. Just he directory structure is enough. It is also very easy to just compile/package one component. Step into the right directory and execute mvn clean install and you are done. I can imagine that the scripts and ant files become a little more complicated to change, but I think it is worth the effort to get a cleaner project layout. I did not have a good look at the Jetty runner yet. But I see it is a class with a very large main method. With maven it is not hard to make an executable jar file. Together with the assembly plugin and an overlay of the original crawler this should not be to hard to create. The integration tests can be put in one project to store them all, but you can also place the integration tests into the project that it is actually testing. Maven has a lifecycle phase for integration tests. Not sure that for instance the FileSystems tests are actually doing. What to they need to run? I see they have a requirement for multiple war files. So it seems logical to me to put them in a separate module. Give them dependencies on the framework modules and run them of you are executing the integration-test phase of maven. The environment related projects can be executed like they are executed now using ant. We can for instance create one maven module on top of them that executes these builds together with the package phase of maven. We could also provide a profile that the build is only executed if you explicitly tell it to. I hope this makes sense to you. Sometime it is hard to explain for me because I am so used to using maven nowadays that I might overlook something why others think it is hard to use. > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie > Attachments: move-to-maven-acf-framework.patch, Screen shot > 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902834#action_12902834 ] Jettro Coenradie commented on CONNECTORS-92: Very cool > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie > Attachments: move-to-maven-acf-framework.patch, Screen shot > 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903363#action_12903363 ] Jettro Coenradie commented on CONNECTORS-92: I did a quick check and it seems to be fine. If you commit the connector changes as well I'll try to commit a patch for both tonight. > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie > Attachments: move-to-maven-acf-framework.patch, Screen shot > 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903460#action_12903460 ] Jettro Coenradie commented on CONNECTORS-92: I see you have created two modules for the api-service. Is there a good reason for that? The servlet is not within the web project. In my original idea I created two modules, but maybe you have a good reason to create two? > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie > Attachments: move-to-maven-acf-framework.patch, Screen shot > 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903462#action_12903462 ] Jettro Coenradie commented on CONNECTORS-92: Wouldn't it be better to rename the *-servlet into something like war or web. There will probably be more things in there than a servlet. > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie > Attachments: move-to-maven-acf-framework.patch, Screen shot > 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903476#action_12903476 ] Jettro Coenradie commented on CONNECTORS-92: Oke, thats fine. But is the projects *-servlet a war? is that the web project? > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie > Attachments: move-to-maven-acf-framework.patch, Screen shot > 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-92: --- Attachment: maven-poms-problem-starting-jetty-and-derby.patch This is a patch that addes poms to the framework part. It runs after you have installed the one missing jar. In the end the mvn clean install should be successful in the root of the framework directory In the crawler-ui project you can do : mvn clean jetty:run-war This will start up the crawler-ui, only after you have copied the conenctors that you need to the src/config/local/connector-lib Than asking for a list of connectors results in a database error. I just cannot get the Derby instance to run. The database is created but if seems not to be running. Any help in this area would be appreciated. Caused by: ERROR 42Y07: Schema 'ACF' does not exist at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSchemaDescriptor(Unknown Source) at org.apache.derby.impl.sql.compile.QueryTreeNode.getSchemaDescriptor(Unknown Source) at org.apache.derby.impl.sql.compile.QueryTreeNode.getSchemaDescriptor(Unknown Source) at org.apache.derby.impl.sql.compile.FromBaseTable.bindTableDescriptor(Unknown Source) at org.apache.derby.impl.sql.compile.FromBaseTable.bindNonVTITables(Unknown Source) at org.apache.derby.impl.sql.compile.FromList.bindTables(Unknown Source) at org.apache.derby.impl.sql.compile.SelectNode.bindNonVTITables(Unknown Source) at org.apache.derby.impl.sql.compile.DMLStatementNode.bindTables(Unknown Source) at org.apache.derby.impl.sql.compile.DMLStatementNode.bind(Unknown Source) at org.apache.derby.impl.sql.compile.CursorNode.bindStatement(Unknown Source) at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source) at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source) at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source) ... 4 more > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie > Attachments: maven-poms-problem-starting-jetty-and-derby.patch, > move-to-maven-acf-framework.patch, Screen shot 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904214#action_12904214 ] Jettro Coenradie commented on CONNECTORS-92: I am afraid the crawler-ui is not done yet. I think we are missing libraries. What I was trying to do is make it run and see what I was missing. But the way I understand it now, I must choose another path to make it run. This seemed the most easy thing to do. I am still thinking about why this is so hard. Would be nice to have something like a servlet or filter that initializes everything that you do in your special runner now. But if you spend a lot of time getting it right, it must not be the easiest thing to do. I will come with a better patch, hope to be able to build something that I can actually test using the sample. > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie > Attachments: maven-poms-problem-starting-jetty-and-derby.patch, > move-to-maven-acf-framework.patch, Screen shot 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-92: --- Attachment: maven-poms-including-start-jar.patch This patch creates all the framework jars and wars as well as a start.jar in the jetty-runner project. The jars need to be in the same location as in the ant generated jar file. The lib folder. Later on we can create a module that grabs all libraries and wars, puts them in the right directory structure. > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie > Attachments: maven-poms-including-start-jar.patch, > maven-poms-problem-starting-jetty-and-derby.patch, > move-to-maven-acf-framework.patch, Screen shot 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904561#action_12904561 ] Jettro Coenradie commented on CONNECTORS-92: As for the start.jar I do not see a problem. I think I am almost there. THe classpath already contains the lib part, so I only need to add the dependencies to the jetty runner project. As for the example code, I do not mind to keep using and to create the example. I only wanted to have maven to make it easier to setup my development environment and to do the dependency management. I'll try to come up with an improve pom for the start.jar, if that is not enough please let me know. > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie >Assignee: Karl Wright > Attachments: maven-poms-including-start-jar.patch, > maven-poms-problem-starting-jetty-and-derby.patch, > move-to-maven-acf-framework.patch, Screen shot 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906778#action_12906778 ] Jettro Coenradie commented on CONNECTORS-92: No, time is my enemy at the moment > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie >Assignee: Karl Wright > Attachments: maven-poms-including-start-jar.patch, > maven-poms-problem-starting-jetty-and-derby.patch, > move-to-maven-acf-framework.patch, Screen shot 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907358#action_12907358 ] Jettro Coenradie commented on CONNECTORS-92: I worked on it tonight but I decided to stop. This path is not leading in a direction that I would like. To make most out of maven I would like to change more than you would be willing to right now. I cannot blame you, because you have something working right now. Maybe someone else wants to step in and finish what I have done. I can submit another patch with the stuff I have right now. > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie >Assignee: Karl Wright > Attachments: maven-poms-including-start-jar.patch, > maven-poms-problem-starting-jetty-and-derby.patch, > move-to-maven-acf-framework.patch, Screen shot 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-92: --- Attachment: maven-start-jar.patch patch containing a good start to create the start.jar from the jetty-runner project > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie >Assignee: Karl Wright > Attachments: maven-poms-including-start-jar.patch, > maven-poms-problem-starting-jetty-and-derby.patch, maven-start-jar.patch, > move-to-maven-acf-framework.patch, patch-connectors.zip, Screen shot > 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jettro Coenradie updated CONNECTORS-92: --- Attachment: patch-connectors.zip Not a patch, but a zip containing a few poms for the connectors. The null connector and the jdbc connector have a pom > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie >Assignee: Karl Wright > Attachments: maven-poms-including-start-jar.patch, > maven-poms-problem-starting-jetty-and-derby.patch, maven-start-jar.patch, > move-to-maven-acf-framework.patch, patch-connectors.zip, Screen shot > 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-92) Move from ant to maven or other build system with decent library management
[ https://issues.apache.org/jira/browse/CONNECTORS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907908#action_12907908 ] Jettro Coenradie commented on CONNECTORS-92: I think maven is not the right tool at the moment due to the libraries that are used that are not available in any repository, the way the sample is started. The dependencies that seem to be copied to each project. I am spending a lot of time on creating an assembly, but that is not really the part where maven shines. > Move from ant to maven or other build system with decent library management > --- > > Key: CONNECTORS-92 > URL: https://issues.apache.org/jira/browse/CONNECTORS-92 > Project: Apache Connectors Framework > Issue Type: Wish > Components: Build > Reporter: Jettro Coenradie >Assignee: Karl Wright > Attachments: maven-poms-including-start-jar.patch, > maven-poms-problem-starting-jetty-and-derby.patch, maven-start-jar.patch, > move-to-maven-acf-framework.patch, patch-connectors.zip, Screen shot > 2010-08-23 at 16.31.07.png > > > I am looking at the current project structure. If we want to make another > build tool available I think we need to change the directory structure. I > tried to place a suggestion in an image. Can you please have a look at it. If > we agree that this is a good way to go, than I will continue to work on a > patch. Which might be a bit hard with all these changing directories, but > I'll do my best to at least get an idea whether it would be working. > So I have three questions: > - Do you want to move to maven or put maven next to ant? > - Do you prefer another build mechanism [ant with ivy, gradle, maven3] > - Do you have an idea about the amount of scripts that need to be changed if > we change the project structure > The image of a possible project layout (that is based on the maven standards) > is attached to the issue -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
