Re: [vote] Release Mojo webstart plugin 1.0-alpha-1
oops Damn it. J On 10/23/06, Jerome Lacoste [EMAIL PROTECTED] wrote: Hi, I'd like to release the first alpha release of the webstart plugin. Plugin has been almost unchanged since before the summer. I intend to change some things before 1.0, but there's clearly a need for an official non-snapshot release of the plugin. The latest snapshot was just deployed (20061023.145534-2) and is available in the Codehaus snapshots repository ([1]). Please vote for the release 1.0-alpha-1 of webstart plugin: [+1]: [0]: [-1]: (obviously +1 from me.) Jerome [1] http://snapshots.repository.codehaus.org/org/codehaus/mojo/webstart-maven-plugin -- Jerome Lacoste - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[vote] Release Mojo webstart plugin 1.0-alpha-1
Hi, I'd like to release the first alpha release of the webstart plugin. Plugin has been almost unchanged since before the summer. I intend to change some things before 1.0, but there's clearly a need for an official non-snapshot release of the plugin. The latest snapshot was just deployed (20061023.145534-2) and is available in the Codehaus snapshots repository ([1]). Please vote for the release 1.0-alpha-1 of webstart plugin: [+1]: [0]: [-1]: (obviously +1 from me.) Jerome [1] http://snapshots.repository.codehaus.org/org/codehaus/mojo/webstart-maven-plugin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [DBCP] state of DBCP
On 6/21/06, Phil Steitz [EMAIL PROTECTED] wrote: On 6/21/06, jerome lacoste [EMAIL PROTECTED] wrote: Hi, I've submitted a patch some days ago and am waiting for comments (DBCP-175). In the mean time, I've had a look at the status of DBCP. Some notes: - no release since 2004 - there are 10+ issues marked UNRESOLVED but with a Resolved status in Jira. Could be cleaned up. - althought I don't have many problems with DBCP myself, I've seen issues of various types in Jira: NPEs, leaks, thread related issues, class loading issues, equals()/hashcode() issues etc... Pretty scary to me :) So I think it is time to decide what to do next... Should there be a 1.2.2 release? A 1.3? Who's involved today in DBCP ? I have been slowly plowing through the issues and have posted a release plan on the wiki here: http://wiki.apache.org/jakarta-commons/DBCP/1%2e2%2e2ReleasePlan OK I am relieved :) I can help triaging those issues, sending patches to fix some of them, but I don't want to send patches that will stay in Jira for years. Please, please keep the patches coming. I am reviewing the ones you submitted and will update Jira / apply in the next day or two. Any one else interested and knowledgeable of [dbcp] code base, pls chime in. Also, pls update the Wiki page to reflect any opinions that you have on which bugs absolutely must get fixed in 1.2.2. Your plan sounds very good. Only question is why not using Jira in the first place to target issues? DBCP-128 should probably be fixed later as it will probably require to change the implementation of equals and hashcode. You never know what this might break... DBCP-68 was committed. See http://issues.apache.org/jira/browse/DBCP-68#action_12411844 Cheers, Jerome - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [DBCP] state of DBCP
Your plan sounds very good. Only question is why not using Jira in the first place to target issues? Because we started planning this release before the Jira migration ;-) Probably a good idea to just do it all in Jira now. Feel free to jump in. Probably can't do much with Jira in terms of editing the fix version field as I have no rights. IANAC(ommiter) :) DBCP-128 should probably be fixed later as it will probably require to change the implementation of equals and hashcode. You never know what this might break... Yes, that is really too bad, but I agree. DBCP-68 was committed. See http://issues.apache.org/jira/browse/DBCP-68#action_12411844 Good. Comments on other tickets and patches welcome! I looked at your other Recommended dispositions in the wiki page and they all seemed logical. Jerome - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[DBCP] state of DBCP
Hi, I've submitted a patch some days ago and am waiting for comments (DBCP-175). In the mean time, I've had a look at the status of DBCP. Some notes: - no release since 2004 - there are 10+ issues marked UNRESOLVED but with a Resolved status in Jira. Could be cleaned up. - althought I don't have many problems with DBCP myself, I've seen issues of various types in Jira: NPEs, leaks, thread related issues, class loading issues, equals()/hashcode() issues etc... Pretty scary to me :) So I think it is time to decide what to do next... Should there be a 1.2.2 release? A 1.3? Who's involved today in DBCP ? I can help triaging those issues, sending patches to fix some of them, but I don't want to send patches that will stay in Jira for years. Cheers, Jerome - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (DBCP-175) [dbcp] I'd like to run init SQL after JDBC Connection creation in Tomcat DBCP
[ http://issues.apache.org/jira/browse/DBCP-175?page=all ] Jerome Lacoste updated DBCP-175: Attachment: DBCP-175-2.txt New version of the second patch that allows to run multiple init statements. The patch now includes a change to the JOCLContentHandler to handle array, collection and list. Not that I needed them all, but the code paths are mostly identical, so why not? With that patch, one can add multiple statements in a jocl file using: collection string value=/ string value=/ /collection Patch is against 1.2.2-SNAPSHOT. Change is unit tested and documented. [dbcp] I'd like to run init SQL after JDBC Connection creation in Tomcat DBCP - Key: DBCP-175 URL: http://issues.apache.org/jira/browse/DBCP-175 Project: Commons Dbcp Type: Improvement Versions: 1.2 Final Environment: Operating System: other Platform: Other Reporter: Jiri Melichna Priority: Minor Attachments: DBCP-175-1.txt, DBCP-175-2.txt, DBCP-175-2.txt, dbcp_add_init_sql.zip Hi! I'm porting some j2ee web based applications from BEA Web Logic to Tomcat 5.5. In BEA Web Lobic connection pool it is possible to specify init SQL qurery that server runs after connection is created (before first use). It's very good for example of national settings in Oracle. For correct czech sorting i have to run setting query: ALTER SESSION SET NLS_SORT = XCZECH First time i tryed to write interceptor into my application. This interceptor runs ALTER SESSION SET NLS_SORT = XCZECH before evey sorted query (SELECT ... ORDER BY...), but i had some performance problems. So i starded to work with DBCP. I was very wandered about DBCP packages in Tomcat 5.5 and i did not find sources of naming-factory-dbcp.jar. So i refactored, enhanced and compiled full DBCP (with pool and collections). Now i'm a little afraid of some library conflicts (full DBCP, full Pool and full Collections refactored into tomcat packages), but performance is OK and basic tests of my application seems good. I added property connectionInitSql for int SQL into BasicDataSource. I had to mofify BasicDataSourceFactory for correct setting of this property. Property is used in method createConnection() of DriverConnectionFactory to init Connection. It would be very nice if you will add this init feature into Tomcat DBCP. Best regards Jiri Melichna -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (DBCP-189) [dbcp] Threads do not get a Connection in FIFO mode
[ http://issues.apache.org/jira/browse/DBCP-189?page=all ]
Jerome Lacoste updated DBCP-189:
Attachment: DBCP-189.txt
Patch that implements the requested changes. Only unit tested. Didn't add a
regression test.
[dbcp] Threads do not get a Connection in FIFO mode
---
Key: DBCP-189
URL: http://issues.apache.org/jira/browse/DBCP-189
Project: Commons Dbcp
Type: Bug
Versions: 1.2 Final
Environment: commons-dbcp-1.2.1.jar
Reporter: rod
Attachments: DBCP-189.txt
hi,
the SharedPoolDataSource class uses the class GenericKeyedObjectPool which
has a FIFO behavior.
the problem is that when all connections in the pool are used, the
synchronised method SharedPoolDataSource.getPooledConnectionAndInfo
calls the borrowObject() blocking method and does not release its monitor.
as a result, all other threads asking for a connection get blocked trying to
get the monitor
and will get a connection later in a non-FIFO mode.
i think the fix is to synchronized only the code block :
if (pool == null) {
try {
registerPool(username, password);
} catch (NamingException e) {
throw new SQLNestedException(RegisterPool failed, e);
}
}
instead of synchronizing the whole method.
thanks
rodolphe
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (DBCP-175) [dbcp] I'd like to run init SQL after JDBC Connection creation in Tomcat DBCP
[ http://issues.apache.org/jira/browse/DBCP-175?page=all ] Jerome Lacoste updated DBCP-175: Attachment: DBCP-175-1.txt DBCP-175-2.txt DBCP-175-1.txt is a version of the change that is similar to the original request. It allows one to add a single initialization SQL statement. We tested this patch on a test server yesterday. DBCP-175-2.txt is a version of the change that would allow one to specify more than one initialization statements. That's probably the most flexible solution. I haven't had time to test it yet. I attach it as a sort of RFC and will complete it if you guys think it's the way to go (I do). Both patches come with unit tests and documentation (hope I forgot nothing). Patches made against trunk (1.2.2-SNAPSHOT). [dbcp] I'd like to run init SQL after JDBC Connection creation in Tomcat DBCP - Key: DBCP-175 URL: http://issues.apache.org/jira/browse/DBCP-175 Project: Commons Dbcp Type: Improvement Versions: 1.2 Final Environment: Operating System: other Platform: Other Reporter: Jiri Melichna Priority: Minor Attachments: DBCP-175-1.txt, DBCP-175-2.txt, dbcp_add_init_sql.zip Hi! I'm porting some j2ee web based applications from BEA Web Logic to Tomcat 5.5. In BEA Web Lobic connection pool it is possible to specify init SQL qurery that server runs after connection is created (before first use). It's very good for example of national settings in Oracle. For correct czech sorting i have to run setting query: ALTER SESSION SET NLS_SORT = XCZECH First time i tryed to write interceptor into my application. This interceptor runs ALTER SESSION SET NLS_SORT = XCZECH before evey sorted query (SELECT ... ORDER BY...), but i had some performance problems. So i starded to work with DBCP. I was very wandered about DBCP packages in Tomcat 5.5 and i did not find sources of naming-factory-dbcp.jar. So i refactored, enhanced and compiled full DBCP (with pool and collections). Now i'm a little afraid of some library conflicts (full DBCP, full Pool and full Collections refactored into tomcat packages), but performance is OK and basic tests of my application seems good. I added property connectionInitSql for int SQL into BasicDataSource. I had to mofify BasicDataSourceFactory for correct setting of this property. Property is used in method createConnection() of DriverConnectionFactory to init Connection. It would be very nice if you will add this init feature into Tomcat DBCP. Best regards Jiri Melichna -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[dbcp] RFC adding initialization statements after connection creation
Hi, Adding a way to run initialization statements after connection creation allowed us to cleanly solve some Oracle sorting issues. This issue was in fact independently reported last year (DBCP-175). I've attached 2 patches to Jira and would appreciate feedback, in particular regarding the second one that would allow to have more than one initialization statements. Do we want to go that way? Please review/comment, Thanks! Jerome Lacoste http://issues.apache.org/jira/browse/DBCP-175 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: How can I search the mailing list archive
On 6/17/06, Kenneth Xu [EMAIL PROTECTED] wrote: Hello, I apologize if this is a stupid question. I know there is an archive but I didn't find a way to search it. I have tried to request for info and FAQ but none available. [EMAIL PROTECTED] [EMAIL PROTECTED] Thanks in advance! try this http://marc.theaimsgroup.com/?l=jakarta-commons-devr=1w=2 Jerome - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [FILEUPLOAD] RfC: Proposed API changes for streaming
On 6/16/06, Jochen Wiedmann [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: As I understand it, by the time you call getInputStream(), the user's POST request is already entirely in the server's memory space, or it has been written to disk. This data isn't consumed when you read() it, so why can't you get another InputStream over it? No. At the time when getInputStream() is called, the request data is read up to the beginning of the file, or form field. The InputStream returns the file, or form field, after which the request data is positioned at the end of the file, or form field. (Caution I am not familiar with the original code. Just read that thread). To me it sounds like SlimFileItem is implying that getInputStream() is doing some kind of iterator side effect. First if kept like this, it probably shouldn't be a getter, maybe inputStream(). Also it makes wonder if the separation between the iterator and the iterator item is done at the right place. Think about JDBC or Collection. This proposal is not intuitive. Second while SlimFileItem should throw an IllegalStateException on the second invocation. Should throw as part of an interface implies that FileItem does not respect the spec of its base class. May throw sounds better. Also the above sentence sounds classish not interfaceish (i.e. reading it makes it sound like SlimFileItem is a class not an interface). Finally, FileBaseUpload would be changed to use getItemIterator internally and convert the SlimFileItem instances into FileItem. I don't particularly like down castings. That ties the class to the particular implementation. What does this hide? Cheers, Jerome - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [exec] design goals, API sketch, etc.. next step?
On 1/24/06, Brett Porter [EMAIL PROTECTED] wrote: jerome lacoste wrote: The refactoring ended up here: http://moca.dynalias.com/~jerome/projects/exec2/ (Unfortunately it seems like I messed up apache access rights, but I should make this code available tomorrow). Let us know, I'll take a look. I don't think I ever saw this... Seems like trac URL rewriting is messing up with my file serving functionality. I made an archive of the refactored package located at: http://moca.dynalias.com/~jerome/commons-exec-refactored_09_2005.tar.bz2 J - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [exec] vision for the library
On 1/24/06, Brett Porter [EMAIL PROTECTED] wrote: jerome lacoste wrote: I'm big on interfaces for defining and maintaining a simple public contract of the API, They help to some extent, but if you are going to use interface just to define your public contract, I think it might be an overuse. An 'implicit' interface is sometimes enough. [...] However, this situation would have to be worked around by adding to the factory instead of replacing. WDYT? Factories have their use, e.g. when you want to keep control on the returned objects, when you can do smart stuff inside the factory to return the more appropriate object and make the client life simpler, etc. I think that today, full fledged dependency injection fits better than this 'old school' IoC pattern. In particular a factory added without real good reasons could limit the client dependening on possibilities. I've seen people go from one class to one class + its interface + one factory [+ the factory interface + ...]. I did that mistake before, and I found out it didn't bring me that much benefits. I am trying to avoid it now. WRT intefaces, I am OK to use one, if it is minimal and has a good use case. In fact if there's no need to have an interface right now, it can always be added afterwards. Same for the factory. In our case, an interface should be minimal. As you said in another post getters and setters sshould not be part of the interface. Basically, we only need something like: Process start() [throws ExecuteException]; But we can add that later on, if needed. So let's design the thing as simple as possible and to grow from there. WDYT? Jerome - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[exec] design goals, API sketch, etc.. next step?
Niklas, Brett, Trygve all, Things were stagnating a little bit end of last year. I am happy to see that things are in motion again. Last interesting initiative was Brett's idea to start with an API sketch. I think it is a good start but it's only solving the start Process feature. That's the low level building block. We ought to provide more (otherwise we are just JDK 1.5 equivalent and that's not interesting to me). See my past mails (expesially 'vision') for more info. Below is a list of past emails that contain information relative to the subject. [exec] Design of commons-exec http://thread.gmane.org/gmane.comp.jakarta.commons.devel/69584 I listed there a list of high level requirements that I want to keep in mind when designing the library. [exec] API, implementation various notes http://thread.gmane.org/gmane.comp.jakarta.commons.devel/72242 Those mails are maybe too long to read, if you can only read one, pick the second in the thread. There I compared the various APIs (JD 1.4, JDK 1.5, old common-exec). This is an important point to remember when making the API sketch. After those mails I refactored the existing commons-exec code according to the vision I had proposed. I also did integrate the refactored library into the CruiseControl code (which has about 30 code points that can make use of commons-exec). The refactoring ended up here: http://moca.dynalias.com/~jerome/projects/exec2/ (Unfortunately it seems like I messed up apache access rights, but I should make this code available tomorrow). [Someone told me at some point that I should send this refactoring as a serie of patches. I can do it now if we think it is worth doing.] This code is important to keep in mind. It keeps the past working code, it has a cleaner interface (although not perfect) and it is integrated with an existing project. To me it's better than an API sketch (although the API sketch is a very nice complement, because it is fresh and untangled). After that I sent a mail based on my experience from the refactoring. [exec] vision for the library http://thread.gmane.org/gmane.comp.jakarta.commons.devel/72295 (once again a long read...) [Exec] API sketch http://thread.gmane.org/gmane.comp.jakarta.commons.devel/77425 And now the latest emails related to the new API sketch. So here's what I propose to do: - decide which requirements are important. What is the problem we are trying to solve? How do we position us compared to the official's API? etc... Someone using SDK 1.4 or 1.5 should find a compelling reason to migrate to commons-exec. The migration should be intuitive. Benefits for migrating clear. I think I exposed my ideas in the thread about the library's vision. - we compare the clean API sketch to the refactoring I made in September. We focus on making this API easy to use and remember without compromise on flexibility. - we then take a decision on how to reach an implementation that satisfies this API (Either start from the API sketch, or from the refactorings I made 4 months ago). I don't care on the approach as long as we get things moving and that efforts are not wasted. Niklas you said you could coordinate. I offered my help for coding, documentation and tests. There's already a lot of work done on that. Who else is interested? How do we proceed? Niklas I let you answer those questions. Let's try not to loose the momentum. I just don't want my efforts to be wasted (so far they haven't been productive enough to my taste). I would really like this library to be in early alpha by end of February. So I can get it off the list of things I said I would do. Cheers, Jerome - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [exec] API sketch?
On 1/13/06, Brett Porter [EMAIL PROTECTED] wrote: Hi, Back in September 05 (wow, is it really 2006 already? :) Jerome sent a comprehensive vision for exec. I found myself agreeing with most everything in there (I'll reply to a couple of specifics shortly). Message ID from then: [EMAIL PROTECTED] However, it was long and wordy. I think the key part was that we should have a minimal exposed API. Maybe a good step forward is to sketch that API, then write implementations that reuse existing code, then refactor code behind the scenes later? BTW, I had implemented my proposal (refactoring the original code) and the main API looked a lot like what was discussed in this thread. Unfortunately, since I write it, I've had 2 disk failures, so I am not sure that code is still accessible. I will try to dig it up to compare it to the API proposed by Niklas. Jerome - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [exec] API sketch?
On 1/17/06, Niklas Gustavsson [EMAIL PROTECTED] wrote: Hi jerome lacoste wrote: I had some changes in mind compared to my original design proposal, to make it simpler. So I am willing to help you design test this API/implementation and make sure it works with CruiseControl. Great! I don't want to duplicate efforts so I would prefer a coordinator on that. Should I take back my original proposal and clean it up? Should we start from scratch? I'll be happy to coordinate the efforts here. Did you have time to look at the quick draft I sent out last week in this thread? When it comes to designing the API I would be happy to start from scratch if that makes it any easier for the users of the library. I think we should strive for a simple API, in line with the XOM design principles [1]. I especially would like us to follow As simple as it can be and no simpler, There's exactly one way to do it and Do not allow clients to do bad things. I don't think the current code fits with any of these so there is some work to be done. I agree with the principles. There's only one issue: backward compatibility. I have over 1000 lines of code that use the former API. So if the switch could be not too hard to do... Whatever the proposal, I want to see code examples to see how it would work in practise. Agreed, what would be the requirements for use within CC? More or less what was covered in my original API description document. My current need is to: - externalize the current Process execution to your library (reduce code in CC) - get more control on the spawned processes to be able to give the user a chance to kill them when something goes wrong. J - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [exec] API sketch?
On 1/13/06, Brett Porter [EMAIL PROTECTED] wrote: Hi, Back in September 05 (wow, is it really 2006 already? :) Jerome sent a comprehensive vision for exec. I found myself agreeing with most everything in there (I'll reply to a couple of specifics shortly). Message ID from then: [EMAIL PROTECTED] However, it was long and wordy. I think the key part was that we should have a minimal exposed API. Maybe a good step forward is to sketch that API, then write implementations that reuse existing code, then refactor code behind the scenes later? With this, we could get something up and working *reasonably* quickly that is *reasonably* stable. Jerome, Niklas, Trygve - anyone interested in picking this up? In some ways the above email described it and this is just an interface sketch. Cool. I was coming back to this subject now as I am more or less done with my other things that took my away for the last 3 months. I had some changes in mind compared to my original design proposal, to make it simpler. So I am willing to help you design test this API/implementation and make sure it works with CruiseControl. I don't want to duplicate efforts so I would prefer a coordinator on that. Should I take back my original proposal and clean it up? Should we start from scratch? Whatever the proposal, I want to see code examples to see how it would work in practise. Jerome - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [exec] vision for the library
Proposed architecture - Launchers: exist as today - OS functionality - ProcessLauncher, (our ProcessBuilder) - Execute (or Executor?): Flexible, we use it to tie together the various concepts in the execute() call (stream management, process management, process building, etc...) - Exec (or ??) makes it easy to use Execute/or. I'm missing the difference between Exec and Executor - but otherwise agree. Exec was deemed to be an object that makes it easy to use the Executor for simple use cases, while Executor was more flexible. The architecture doesn't need to be very different from what it is today. In fact I think it can be achieved by refactoring the current code. I like that. I'd like to have some clean interfaces even if they are similar. I think we should use interfaces with an implementation so we can later do what Dion suggested at one point - a remote exec implementation. I'd rather have implementations then we add an interface in a later version if we need a remote exec. Jerome, trying to use less interfaces - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [exec] vision for the library
On 10/9/05, Niklas Gustavsson [EMAIL PROTECTED] wrote: Hi We regard to my other patches, they can be discarded if we go for a rewrite (through refactorings). Trygve: in particular the serializable patch is not critical. Was just there to maintain the functionality of the old API, as I thought it as critical as that time I started looking at the project. Would it be possible to break your refactorings down into a set of patches that each fix on of the issues you outlined above? Sorry for missing your mail. I don't read this mailing list everyday, and I didn't expect someone to come back to that mail :) I can do that. I have some things with higher priority now related to m2, but as soon as I am done with it, I go back to this lib. Thanks for the feedback. Cheers, Jerome - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[exec] vision for the library
[Sorry for the long mail. I hope I was clear enough this time] This complements/summarizes the couple of mails I sent this week-end, trying to give a better description of the vision I have for commons-exec. I don't want to impose anything. A library like commons-exec is something I have been thinking over for several months, so I am just a little bit enthousiastic when I saw this project getting started in commons. Approach - My approach was the following. Brett asked me to make a small how-to. So I wrote the little how-to, looking at the API and how it was implemented. I compared using commons-exec as of today and what the different alternatives (in particular SDK 5.0) had for them and how they had to be used. Doing so I read all the code and found some things that could be improved. I first started by annotating the code with @todo (see #36798 [1]). I know it's not the correct approach (and I didn't expect the patch to be checked in), but it helps a lot to mark locally each piece of code with a particular issue. So it was an easy way to share the problems. Then I sent a mail with a short vision and described how I saw the library evolving, and did that refactoring just to prove (also to me) that those ideas could fit together. I stopped at a certain point so the current implementation doesn't contain all my ideas yet. I tried refactored the initial code (instead of rewriting it) to not lose the information it had gathered around the years. Not sure if I broke stuff as there wasn't so many unit tests :) NOTE: I didn't take into account whether this library should do more than what it needs today. I am mostly satisfied with the current set of features. But. Current issues There's a certain legacy in the current commons-exec and it shows a lot. The main issue is the following: commons-exec as today was written to solve a particular problem within ant and not as a reusable library. In particular, reusing it forces any client to reuse many classes that a client should not necessarily care about. In comparison ProcessBuilder (SDK 5.0) (which solves only a small subset of what commons-exec solves), is much cleaner in its approach. It relies much more on SDK classes: no need for a CommandLine class, no need for an Environment class, etc... In commons-exec the Watchdog and the ProcessDestroyer have their own issues. And the functionality of some things are not open enough for reuse. For some of these issues, see the notes in my patch [2] and in the aforementionned mails. Let me know if this lacks details. I see commons-exec as an not only an improvement of what Runtime.exec() brings, but as what ProcessBuilder brings. To me, commons-exec should close the following holes let by ProcessBuilder: - ProcessBuilder is too dependent on native calls under the hood. We have a pure java implementation (that depends on some native calls exposed by the SDK). - can be used on SDK 5.0 - can be upgraded. If I find an issue in SDK 5.0, I don't want to wait for a point release to be able to make it work. commons-exec functionality is so important that I don't want to depend on the SDK for this feature. - adds process management capabilities. - adds streams management capabilities (ProcessBuilder only adds the stream redirector stuff, which I think is a strange addition as that level of the API. We can see that later on). So commons-exec has the good functionality, it needs to be more flexible simpler to use. Current commons-exec features It - starts commands in the various platforms and SDK. Todays that's the launcher package. It's mostly good as is. - retrieves os environment, determining on which os we are, etc.. Today that's the OS class + environment package. - builds a process (that's part of Execute) - attaches stream and process operations around the execution of a Process (still part of Execute). - tries to provide good defaults for executing a process (hides complexity of the Execute) (What Exec does today). - should allow clients to make their own stream operations (redirection, visitor-like patterns for consuming information, etc...). Today exists but not that reusable. In CruiseControl I use Execute but don't reuse Exec. That shows its limits. ProcessBuilder like class --- ProcessBuilder is a java.lang class. This means it's a core functionality of the SDK. It solves a simple problem: how to prepare and start processes in a reusable way. It's API makes use of String and Map (which is in an interesting thing, because it's one of the rate classes in java.lang that makes use of out of package classes, and create a dependency loop). Why introducing a ProcessBuilder like class in commons-exec? + design: Today Execute does everything. It can be split in 2 and focus on the execution (i.e. , not on the prepartion of the execution. It can in fact easily be extracted from the current Execute (as
Re: [exec] API, implementation various notes
Proposal to me commons-exec is slightly over-engineered. As is, clients will have to resort to strange things and rewrite somewhat complex helper classes to do simple things. I propose to change the API. Vision: Make commons-exec a reusable Library that is available to all SDK, that not only solves the problems solved by ProcessBuilder but also add functionality for Process and Stream management, allowing flexible reuse of this functionality. [blablabla...] better to show code than words? I started refactoring the API to something that fits better the vision I have with this library. I would appreciate if you could review some of the design ideas I put in this refactoring. I can then write send patches to migrate from the original version to something we agree with. Here's a list of some of the things I did or plan to do in this refactoring: DONE Renamed environment package to os Moved OS into os package Made Environment simpler Made Environment a real Map Made Environment values the String value and not EnvironmentVariable instances Moved getProcEnvironment in separate ProcEnvironment class Renamed getProcEnvironment to getenv() Made CommandLauncher use Map instead of Environment - moved conversion of Environment to Map inside abstract command launcher class (for now) Introduced ProcessLauncher (our ProcessBuilder for SDK 5.0) Made Execute delegate to ProcessLauncher Made Execute and Exec use Map instead of Environment in their API Merged ProcessDestroyer and Watchdog behind a unique Execute.Listener interface TODO: - finish Execute - look at Exec - look at stream management - implement support for SDK 5.0 - constants The files can be viewed at: http://moca.dynalias.com/~jerome/projects/exec2/ Cheers, Jerome PS: first time I participate in the commons project, so let me know if there's something I don't do right. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[exec] API, implementation various notes
Hi, I did the first step at integrating commons-exec into CruiseControl. It's not in the trunk yet [1]. I am not using all the functionality (Exec) and we still have some duplicated code that could be fixed if commons-exec was a little bit more flexible (i.e. add some interfaces where appropriate). I made a number of comments on things that I find strange in the current commons-exec API / implementation. http://moca.dynalias.com/~jerome/projects/commons-exec/patches/ One thing I wonder is about API stability. There are a certain number of small changes I would like to see in commons-exec to make it easier to (re-)use. Things that I would also like to see in commons-exec, is transparent use of ProcessBuilder (SDK 5.0) when available. This SDK class has a funky API anyway, see also http://www.coffeebreaks.org/blogs/?p=17 for some pointers (shameless link to my blog). I am also voluntary to produce documentation for the package (as said to Brett and Trygve yesterday), help redesign things if these small redesigns are allowed, making sure if will be used fully in CruiseControl and why not help ant migration when it comes. Where do we start discussing all these issues? Cheers, Jerome [1] http://moca.dynalias.com/~cruisecontrol/projects/cruisecontrol-trunk/patches/patches/refactor_commons_exec.diff - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [exec] API, implementation various notes
I am also voluntary to produce documentation for the package (as said
to Brett and Trygve yesterday), help redesign things if these small
redesigns are allowed, making sure if will be used fully in
CruiseControl and why not help ant migration when it comes.
Some notes below on that regard:
Evaluation of the current commons-exec design.
Process + Runtime.exec (SDK 5.0)
--
String[] cmdarray = new String[] { myCommand, myArg1, myArg2};
String[] envp = new String[] {VAR1=myValue};
File workinDir = new File(/tmp);
Process p = Runtime.exec(cmdarray, envp, workingDir);
Limitations/drawbacks:
- no difference between command and arguments
- when envp == null, current environment inherited fully. Not possible
to remove one variable.
- lack handling of streams (because not consuming them properly can
lead to hangs).
- lack handling of processes
- ugly confusing Process API (e.g. getInputStream() returns an input
stream that represents the data piped from the output stream. Yuckk)
- nothing makes it really reusable.
ProcessBuilder (SDK = 5.0)
---
ProcessBuilder pb = new ProcessBuilder(myCommand, myArg1, myArg2);
MapString, String env = pb.environment();
env.put(VAR1, myValue);
env.remove(OTHERVAR);
env.put(VAR2, env.get(VAR1) + suffix);
pb.directory(myDir);
Process p = pb.start();
// must handle streams, kill process etc...
Most of the issues fixed.
Limitations/drawbacks
- streams and Processes still not handled by implementation
- API still ugly (see directory() and command() methods)
- requires SDK 5.0...
commons-exec
current API taken from Ant. Can be reused in several projects (Plexus,
CruiseControl etc...).
Provide beans for easy XML - Java mapping.
Low level code centered around Execute, CommandLine and Environment.
Similar in one way to what ProcessBuilder tries to do.
String[] arguments = new String[] {myArg1, myArg2};
String[] envp = new String[] {VAR1=myValue};
ExecuteStreamHandler streamHandler = new PumpStreamHandler();
ExecuteWatchdog watchdog = new ExecuteWatchdog();
Execute execute = new Execute();
CommandLine commandLine = new CommandLineImpl(cmdarray)
commandLine.setExecutable(myCommand);
commandLine.setArguments(arguments)
execute.setCommandLine(commandLine);
Environment environment = Environment.getProcEnvironment();
environment.remove(OTHERVAR);
execute.setEnvironment(environment);
int result = execute.execute();
Exec: code is broken at best. Tries to hide the complexity of using Execute
Because I find it clumsy I prefered to work directly on top of Execute
when integrating it with CruiseControl.
Exec exec = new Exec();
CommandLine commandLine = new CommandLineImpl(cmdarray)
commandLine.setExecutable(myCommand);
commandLine.setArguments(arguments)
// exec.setSpawn(true);
exec.setTimeout(1);
// exec.setNewenvironment(true); // overrides proc env
// exec.setResolveExecutable(true); //
exec.setDir(new File());
// exec.execute(commandLine); // a simpler version (provides
defaults for unspecified parameters/properties. Cover most cases?)
// but most flexible is
exec.execute(commandLine, environment, OutputStream,
OutputStream); // hide type of failure
Key benefits
+ Execute.execute() does both process management (kill) and stream management.
But what complexity needed to use it! Can't use simple things like
String[] and Map. ga
+ Process is hidden, but could be obtained if we were to add some listeners.
+ Environment.getProcEnvironment(). System.getenv() is SDK 5.0.
+ launchers. But are these used at all? See my TODO_comments.diff patch
Limitations/Drawbacks
- execute make it complex by having both watchdog process destroyer interfaces
- not enough interfaces (ExecuteWatchdog)
- Lacks the concept of builder for reusability?
- do not have the flexibility ProcessBuilder has with Environment
- Exec: executable resolving (broken)
- Exec: converts failures, timeouts to ExecuteException (sub
IOException) inconditionaly.
- forces inheritance to do something else with the streams (which we
definitively want in CruiseControl). Yuck.
Proposal
to me commons-exec is slightly over-engineered. As is, clients will
have to resort to strange things and rewrite somewhat complex helper
classes to do simple things.
I propose to change the API.
Vision: Make commons-exec a reusable Library that is available to all
SDK, that not only solves the problems solved by ProcessBuilder but
also add functionality for Process and Stream management, allowing
flexible reuse of this functionality.
For that I need to look at the forces in Action, in particular look at
how this can be improved without requiring too much change in Ant. I
believe it's possible.
Some ideas to make it simpler:
- make Execute less dependent on classes from
Re: [exec] use of commons-logging
On 9/23/05, Trygve Laugstøl [EMAIL PROTECTED] wrote: On Fri, 2005-09-23 at 10:36 +0700, Kev Jackson wrote: Hi, I've been looking at the use of commons logging in exec. Coming from ant dev, it would be nice to refactor ant to use the commons-exec package, but Ant has it's own logging mechanism (using logging listeners), and I'm pretty sure that the ant code base will not change to using commons-logging throughout except as the listener, and as far as I'm aware the ant team want to keep the ant bootstrap as minimal as possible (require nothing but an xml parser and a jdk). [1] There's also the problem of the minimum version of Java supported by commons-exec, though this looks like it will be less of a problem. I'm concerned about this issue too and have been looking into ways to automate the testing to make sure that we're not using any JDK1.3+ API parts. Do you have any good ideas? I can also set up an automated build that uses 1.3... that should catch errors (a little bit late, but easy to set up). J - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [exec] Design of commons-exec
On 8/6/05, Niklas Gustavsson [EMAIL PROTECTED] wrote: Hi Jerome Thanks for a lot of really good feedback and ideas! I will come back to some of the other stuff later on, but first I think we need to discuss on of the points you bring up: jerome lacoste wrote: * run on the oldest SDK possible I think there are three ways to go: 1. Full support for 1.1+ 2. Limited support for 1.1-1.2, full support for 1.3+ 3. No support for 1.1-1.2, full support 1.3+ go for 3 'oldest' I really meant not SDK 5.0 only. J - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [exec] Design of commons-exec
Thanks a lot to Eric for pointing me to this thread. As he said, I started a similar project some weeks ago and put some notes to share on http://www.coffeebreaks.org/blogs/?p=8. I would have preferred more feedback from the jakarta guys before going further. Looks like my wishes are coming true :) In the proposal, I wrote some requirements and design ideas. Maybe the design ideas are too far from the original Ant classes. But I believe the requirements should still hold for commons-exec. So I paste them below: Requirements features (proposal) * handles the definition of a command line and its running in a flexible framework * minimal external dependencies (0 if possible) * run on the oldest SDK possible * increased thread safety * POJO objects where appropriate * extension friendly * automatic handling of command execution timeout * library should not be in the way of clients. E.g. possibility for clients to monitor or kill underlying processes * test friendly. In particular easy to mock the running of a command line, e.g. replacing Process inputs/outputs with normal streams. Maybe provide some mocks as part of the lib or in a jcli-mock lib. [...] Do not forget to look into the new SDK 1.5 ProcessBuilder class. One of the main issue CruiseControl is having today is in the Process handling. In this server side environment, timeouts are not always practical and we need to give the user more control. So we need a Process Destroyer like facility, not something automated that is executed when the VM terminates, but something that gives more control over the life/death of the processes. Some kind of user triggered watchdog instead of a timeout triggered one. The design ideas I tried to describe on the aforementionned page are focused on trying to solve this issue. They are probably a little bit too complex. Nevertheless feel free to comment on them. Answers to your questions: * Do you think this fits with what you would find appropriate and useful for commons-exec? Sounds good to me. Except the Process destroyer as pointed by others. * Would it be a good starting point (if so I'll create a patch for the code I've cleaned up and removed the Ant specifics from)? a patch or a tarball? Is there some code already somewhere today outside of Ant's repository? * Should I put this on the wiki for further discussions? If you do so, please send the URL. I am not sure if I can contribute directly to this library, but you can be assured that I will use it as soon as there's something usable, to make sure it fixes out issues in CruiseControl. Thanks for taking the initiative! Cheers, Jerome - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
