Re: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
Commit it on a branch. -- Scott Stark Chief Technology Officer JBoss Group, LLC Hiram Chirino wrote: The change is too big for a patch. I'd rather commit on a branch. Another option is to refactor it some more so that it becomes part of the new module that Nathan is working on. Either way, please let me know which way you prefer it. I have completed most of my clean up and the jms and mdb unit tests are once again passing with flying colors. The mailing list does not seem to have received a couple of emails I sent a few days back. Is there something wrong with the mailing list? Regards, Hiram --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps1 ___ JBoss-Development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
Hiram, no one sits around for months without interacting with the day to day developers of JBoss Group and then decides to commit a large change without it being discussed. Some of what you have most likely has merit but it has to be reviewed so either submit it as a patch or commit it to a branch off of head so that it can be reviewed for incorporation. Scott Stark Chief Technology Officer JBoss Group, LLC -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Hiram Chirino Sent: Wednesday, July 09, 2003 12:16 AM To: [EMAIL PROTECTED] Subject: Re: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite Scott, Why does it matter? Nathan has not expressed interested in growing from the current JMS implementation. I've been waiting for several months for the new general purpose implementation to 'appear' and it has not. There has been a skeleton since early Jun. So it's time for me to start the engine again and make some needed improvements to the current JBossMQ implementation. Where have you been the past year when Adrian and Scott have been fixing this buggy MQ implementation? You failed to keep up the old stuff so please don't make it sound like you're coming to the rescue. I'm not so sure I want a total refactoring of the old JMS in the remoting subsystem and the interceptor chains and such. The current JMS rewrite by Nathan, Adrian, and Bela is going quite well and we will be replacing the old system in the fall. What I don't want is a CVS HEAD of the old JMS that doesn't look like the 3.2 series since it will be much harder to migrate 3.2 fixes to a CVS HEAD that has been totally refactored. The old JMS needs to live a bit beyond 4.0's release and 4.0 will not be released until the late late fall, between Thanksgiving and Christmas. I guess what I'm saying is that a lot of users will still depend on the old JMS for at least another year and we need some to have fluidity between 3.2 and 4.0. We're already experiencing the pain of an unnecessary rewrite of the Entity Container that is making it difficult to merge fixes in 3.2 to 4.0. I don't want the same thing to happen with a codebase that is going to be retired eventually anyways and that needs to live in depracated mode for awhile. --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps1 ___ JBoss-Development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
The change is too big for a patch. I'd rather commit on a branch. Another option is to refactor it some more so that it becomes part of the new module that Nathan is working on. Either way, please let me know which way you prefer it. I have completed most of my clean up and the jms and mdb unit tests are once again passing with flying colors. The mailing list does not seem to have received a couple of emails I sent a few days back. Is there something wrong with the mailing list? Regards, Hiram On Fri, 2003-07-11 at 13:01, Scott M Stark wrote: Hiram, no one sits around for months without interacting with the day to day developers of JBoss Group and then decides to commit a large change without it being discussed. Some of what you have most likely has merit but it has to be reviewed so either submit it as a patch or commit it to a branch off of head so that it can be reviewed for incorporation. Scott Stark Chief Technology Officer JBoss Group, LLC -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Hiram Chirino Sent: Wednesday, July 09, 2003 12:16 AM To: [EMAIL PROTECTED] Subject: Re: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite Scott, Why does it matter? Nathan has not expressed interested in growing from the current JMS implementation. I've been waiting for several months for the new general purpose implementation to 'appear' and it has not. There has been a skeleton since early Jun. So it's time for me to start the engine again and make some needed improvements to the current JBossMQ implementation. Where have you been the past year when Adrian and Scott have been fixing this buggy MQ implementation? You failed to keep up the old stuff so please don't make it sound like you're coming to the rescue. I'm not so sure I want a total refactoring of the old JMS in the remoting subsystem and the interceptor chains and such. The current JMS rewrite by Nathan, Adrian, and Bela is going quite well and we will be replacing the old system in the fall. What I don't want is a CVS HEAD of the old JMS that doesn't look like the 3.2 series since it will be much harder to migrate 3.2 fixes to a CVS HEAD that has been totally refactored. The old JMS needs to live a bit beyond 4.0's release and 4.0 will not be released until the late late fall, between Thanksgiving and Christmas. I guess what I'm saying is that a lot of users will still depend on the old JMS for at least another year and we need some to have fluidity between 3.2 and 4.0. We're already experiencing the pain of an unnecessary rewrite of the Entity Container that is making it difficult to merge fixes in 3.2 to 4.0. I don't want the same thing to happen with a codebase that is going to be retired eventually anyways and that needs to live in depracated mode for awhile. --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps1 ___ JBoss-Development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development -- /** * Hiram Chirino * Partner * Core Developers Network **/ --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps1 ___ JBoss-Development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
I prefer it be commit on a branch. Thanks, Nathan -Original Message- From: [EMAIL PROTECTED] [mailto:jboss- [EMAIL PROTECTED] On Behalf Of Hiram Chirino Sent: Friday, July 11, 2003 4:32 PM To: [EMAIL PROTECTED]; [EMAIL PROTECTED]; Jeremy Boynes Subject: Re: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite The change is too big for a patch. I'd rather commit on a branch. Another option is to refactor it some more so that it becomes part of the new module that Nathan is working on. Either way, please let me know which way you prefer it. I have completed most of my clean up and the jms and mdb unit tests are once again passing with flying colors. The mailing list does not seem to have received a couple of emails I sent a few days back. Is there something wrong with the mailing list? Regards, Hiram On Fri, 2003-07-11 at 13:01, Scott M Stark wrote: Hiram, no one sits around for months without interacting with the day to day developers of JBoss Group and then decides to commit a large change without it being discussed. Some of what you have most likely has merit but it has to be reviewed so either submit it as a patch or commit it to a branch off of head so that it can be reviewed for incorporation. Scott Stark Chief Technology Officer JBoss Group, LLC -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Hiram Chirino Sent: Wednesday, July 09, 2003 12:16 AM To: [EMAIL PROTECTED] Subject: Re: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite Scott, Why does it matter? Nathan has not expressed interested in growing from the current JMS implementation. I've been waiting for several months for the new general purpose implementation to 'appear' and it has not. There has been a skeleton since early Jun. So it's time for me to start the engine again and make some needed improvements to the current JBossMQ implementation. Where have you been the past year when Adrian and Scott have been fixing this buggy MQ implementation? You failed to keep up the old stuff so please don't make it sound like you're coming to the rescue. I'm not so sure I want a total refactoring of the old JMS in the remoting subsystem and the interceptor chains and such. The current JMS rewrite by Nathan, Adrian, and Bela is going quite well and we will be replacing the old system in the fall. What I don't want is a CVS HEAD of the old JMS that doesn't look like the 3.2 series since it will be much harder to migrate 3.2 fixes to a CVS HEAD that has been totally refactored. The old JMS needs to live a bit beyond 4.0's release and 4.0 will not be released until the late late fall, between Thanksgiving and Christmas. I guess what I'm saying is that a lot of users will still depend on the old JMS for at least another year and we need some to have fluidity between 3.2 and 4.0. We're already experiencing the pain of an unnecessary rewrite of the Entity Container that is making it difficult to merge fixes in 3.2 to 4.0. I don't want the same thing to happen with a codebase that is going to be retired eventually anyways and that needs to live in depracated mode for awhile. --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps1 ___ JBoss-Development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development -- /** * Hiram Chirino * Partner * Core Developers Network **/ --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps1 ___ JBoss-Development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps1 ___ JBoss-Development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Antwort: RE: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
The current JMS rewrite by Nathan, Adrian, and Bela is going quite well and we will be replacing the old system in the fall. Don't work on a codebase that is going to be retired and needs to live in depracated mode for awhile. A refactoring isn't what is needed in the JMS subsystem. I don't want to be pessimistic, but it took quite a long time to stabilize the current JBossMQ. This is especially true for heavy load scenarios. The ongoing JMS rewrite hopefully will bring significant improvements, but first of all it has to proof stability and spec compliance before it is a valid alternative for production usage. Many JMS users will have to / want to stay with JBossMQ for quite a longer than maybe expected ! Therefore ongoing improvements of the existing JBossMQ code have quite a value ( atleast for me as a JMS user, who requires a stable and matured JMS plattform :-) Regards Ulf
RE: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Hiram Chirino Sent: Wednesday, July 09, 2003 12:16 AM To: [EMAIL PROTECTED] Subject: Re: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite Scott, Why does it matter? Nathan has not expressed interested in growing from the current JMS implementation. I've been waiting for several months for the new general purpose implementation to 'appear' and it has not. There has been a skeleton since early Jun. So it's time for me to start the engine again and make some needed improvements to the current JBossMQ implementation. Where have you been the past year when Adrian and Scott have been fixing this buggy MQ implementation? You failed to keep up the old stuff so please don't make it sound like you're coming to the rescue. I'm not so sure I want a total refactoring of the old JMS in the remoting subsystem and the interceptor chains and such. The current JMS rewrite by Nathan, Adrian, and Bela is going quite well and we will be replacing the old system in the fall. What I don't want is a CVS HEAD of the old JMS that doesn't look like the 3.2 series since it will be much harder to migrate 3.2 fixes to a CVS HEAD that has been totally refactored. The old JMS needs to live a bit beyond 4.0's release and 4.0 will not be released until the late late fall, between Thanksgiving and Christmas. I guess what I'm saying is that a lot of users will still depend on the old JMS for at least another year and we need some to have fluidity between 3.2 and 4.0. We're already experiencing the pain of an unnecessary rewrite of the Entity Container that is making it difficult to merge fixes in 3.2 to 4.0. I don't want the same thing to happen with a codebase that is going to be retired eventually anyways and that needs to live in depracated mode for awhile. Nathan, Your doing great work in the peer based JMS arena. But have you formulated a game plan for the rewrite of general purpose implementation? Right now I'm going down the route of simplifying our current c/s implementation down enough so that we can start taking it apart more easily if required to add the needed features. If you still are up to merging fixes in 3.2 to HEAD that is fine. I'm not sure we want a full refactoring. Bill --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Hiram Chirino Sent: Wednesday, July 09, 2003 12:38 AM To: [EMAIL PROTECTED] Subject: RE: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite On Tue, 2003-07-08 at 23:41, Nathan Phelps wrote: In line... Hiram, As you may know, we are going in a different direction with JMS than the original architecture coded by Norbert Lataille. We are doing a rewrite I guess I had it good. Norbert made a good start. At least basic pub/sub worked. That's better than starting from scratch. so patches to the old JBossMQ have a limited lifetime. That means that changes made to the old JBossMQ will most likely not be part of HEAD or the distribution as we move forward. You may be right. or wrong. The current JMS will ship until there is a better replacement. Do you plan to replace the current implementation with the peer based implementation you have been working on? No, the peer based implementation will only be a flavor. As you already know, multicast can't be used for everything. Bill --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
As you may know, we are going in a different direction with JMS than the original architecture coded by Norbert Lataille. We are doing a rewrite I guess I had it good. Norbert made a good start. At least basic pub/sub worked. That's better than starting from scratch. Enough of this already! You give credit where credit is due. Norbert Lataille did 80% of the work, you and David Maplesden did the 20% needed for a solid release, but the better than starting from scratch doesn't give credit. You may be right. or wrong. The current JMS will ship until there is a better replacement. Do you plan to replace the The current JMS rewrite by Nathan, Adrian, and Bela is going quite well and we will be replacing the old system in the fall. Don't work on a codebase that is going to be retired and needs to live in depracated mode for awhile. A refactoring isn't what is needed in the JMS subsystem. As scott asked you, please make sure you communicate with nathan, the lead on JMS implementation. Otherwise your work will fall by the wayside. Also the forums are a good place to sync up or ask for clarifications. marcf --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
Hi guys, Over the past two weeks I have started to make a few improvements the current JBossMQ implementation that is in CVS HEAD. I would consider a large porting of what I did refactoring to simplify the current code base to allow future growth without having to sacrifice current features or performance. I just wanted to give provide an update on the development status of the original JBossMQ client-server JMS implementation. Here's a summary of what has changed: - It exclusively uses the remoting subsystem for client server communications. The existing IL subsystem has been removed. The 'async' remoting protocol allows two way communications solely over client sockets (this should allow applet/firewalled clients to connect to the server). Multiple JMS connections made by the client will also share a singe pool of sockets to communicate with the server. - Manually creating JBossMQ ConnectionFactory objects have been simplified since a remoting URI takes care setting up most of the connection options. - The server interceptors have been simplified since they now operate on a typeless invocations (it now matches the style of the other jboss interceptors) - The implementations for the p2p and pub/sub JMS API classes have been consolidated since that is the direction that the JMS 1.1 API is taking (being able to mix the us of p2p and pub/sub with in the same session). - Many (almost all) of the bug fixes/performance enhancements made in the 3.2 branch have been ported forward. - All the PMs except JDBC2 have been removed to make it easier to more quickly make enhancements to the persistence manager interface. I hope a cmp/jdo implementation can be created in the future. - the old org.jboss.mq.xml package that provided a simple xml dom was removed and all code that was using was changed to use dom4j instead. - JNDI Reference object handling was simplified (only StringRefs are used). - The sources were refactored so that the back-end messing server is much less coupled the javax.jms.* classes. The server side is still importing many of the JMSException classes, this can be cleaned up in the future. This might start to lead the way to allow this messaging server to support other messaging APIs (JavaSpaces?) Things to watch out for with this new version: - The jboss API to manually create ConnectionFactory and Destination objects has changed. Most users should not be affected since the JMS spec states that they should be using JNDI to get those objects. - Previous JBossMQ clients will not be able to connect to this new server version (since it is using remoting for communications). Those clients need to upgrade their jbossmq-client.jar. - The message format has changed so the new server will not be able to restore messages saved to persistence storage by the new server. I hope to commit the changes CVS in the next few days. I have 3 jms test cases that I still need to fix and then I still need to run the MDB test cases. Regards, Hiram -- /** * Hiram Chirino * Partner * Core Developers Network **/ --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
And what interaction has there been with Nathan who originally responded to the rewrite query? -- Scott Stark Chief Technology Officer JBoss Group, LLC Hiram Chirino wrote: Hi guys, Over the past two weeks I have started to make a few improvements the current JBossMQ implementation that is in CVS HEAD. I would consider a large porting of what I did refactoring to simplify the current code base to allow future growth without having to sacrifice current features or performance. I just wanted to give provide an update on the development status of the original JBossMQ client-server JMS implementation. Here's a summary of what has changed: - It exclusively uses the remoting subsystem for client server communications. The existing IL subsystem has been removed. The 'async' remoting protocol allows two way communications solely over client sockets (this should allow applet/firewalled clients to connect to the server). Multiple JMS connections made by the client will also share a singe pool of sockets to communicate with the server. - Manually creating JBossMQ ConnectionFactory objects have been simplified since a remoting URI takes care setting up most of the connection options. - The server interceptors have been simplified since they now operate on a typeless invocations (it now matches the style of the other jboss interceptors) - The implementations for the p2p and pub/sub JMS API classes have been consolidated since that is the direction that the JMS 1.1 API is taking (being able to mix the us of p2p and pub/sub with in the same session). - Many (almost all) of the bug fixes/performance enhancements made in the 3.2 branch have been ported forward. - All the PMs except JDBC2 have been removed to make it easier to more quickly make enhancements to the persistence manager interface. I hope a cmp/jdo implementation can be created in the future. - the old org.jboss.mq.xml package that provided a simple xml dom was removed and all code that was using was changed to use dom4j instead. - JNDI Reference object handling was simplified (only StringRefs are used). - The sources were refactored so that the back-end messing server is much less coupled the javax.jms.* classes. The server side is still importing many of the JMSException classes, this can be cleaned up in the future. This might start to lead the way to allow this messaging server to support other messaging APIs (JavaSpaces?) Things to watch out for with this new version: - The jboss API to manually create ConnectionFactory and Destination objects has changed. Most users should not be affected since the JMS spec states that they should be using JNDI to get those objects. - Previous JBossMQ clients will not be able to connect to this new server version (since it is using remoting for communications). Those clients need to upgrade their jbossmq-client.jar. - The message format has changed so the new server will not be able to restore messages saved to persistence storage by the new server. I hope to commit the changes CVS in the next few days. I have 3 jms test cases that I still need to fix and then I still need to run the MDB test cases. Regards, Hiram --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
Hiram, As you may know, we are going in a different direction with JMS than the original architecture coded by Norbert Lataille. We are doing a rewrite so patches to the old JBossMQ have a limited lifetime. That means that changes made to the old JBossMQ will most likely not be part of HEAD or the distribution as we move forward. Thanks, Nathan Phelps JMS Lead JBoss Group, L.L.C. -Original Message- From: [EMAIL PROTECTED] [mailto:jboss- [EMAIL PROTECTED] On Behalf Of Hiram Chirino Sent: Tuesday, July 08, 2003 8:41 PM To: [EMAIL PROTECTED] Subject: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite Hi guys, Over the past two weeks I have started to make a few improvements the current JBossMQ implementation that is in CVS HEAD. I would consider a large porting of what I did refactoring to simplify the current code base to allow future growth without having to sacrifice current features or performance. I just wanted to give provide an update on the development status of the original JBossMQ client-server JMS implementation. Here's a summary of what has changed: - It exclusively uses the remoting subsystem for client server communications. The existing IL subsystem has been removed. The 'async' remoting protocol allows two way communications solely over client sockets (this should allow applet/firewalled clients to connect to the server). Multiple JMS connections made by the client will also share a singe pool of sockets to communicate with the server. - Manually creating JBossMQ ConnectionFactory objects have been simplified since a remoting URI takes care setting up most of the connection options. - The server interceptors have been simplified since they now operate on a typeless invocations (it now matches the style of the other jboss interceptors) - The implementations for the p2p and pub/sub JMS API classes have been consolidated since that is the direction that the JMS 1.1 API is taking (being able to mix the us of p2p and pub/sub with in the same session). - Many (almost all) of the bug fixes/performance enhancements made in the 3.2 branch have been ported forward. - All the PMs except JDBC2 have been removed to make it easier to more quickly make enhancements to the persistence manager interface. I hope a cmp/jdo implementation can be created in the future. - the old org.jboss.mq.xml package that provided a simple xml dom was removed and all code that was using was changed to use dom4j instead. - JNDI Reference object handling was simplified (only StringRefs are used). - The sources were refactored so that the back-end messing server is much less coupled the javax.jms.* classes. The server side is still importing many of the JMSException classes, this can be cleaned up in the future. This might start to lead the way to allow this messaging server to support other messaging APIs (JavaSpaces?) Things to watch out for with this new version: - The jboss API to manually create ConnectionFactory and Destination objects has changed. Most users should not be affected since the JMS spec states that they should be using JNDI to get those objects. - Previous JBossMQ clients will not be able to connect to this new server version (since it is using remoting for communications). Those clients need to upgrade their jbossmq-client.jar. - The message format has changed so the new server will not be able to restore messages saved to persistence storage by the new server. I hope to commit the changes CVS in the next few days. I have 3 jms test cases that I still need to fix and then I still need to run the MDB test cases. Regards, Hiram -- /** * Hiram Chirino * Partner * Core Developers Network **/ --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
Scott, Why does it matter? Nathan has not expressed interested in growing from the current JMS implementation. I've been waiting for several months for the new general purpose implementation to 'appear' and it has not. So it's time for me to start the engine again and make some needed improvements to the current JBossMQ implementation. Nathan, Your doing great work in the peer based JMS arena. But have you formulated a game plan for the rewrite of general purpose implementation? Right now I'm going down the route of simplifying our current c/s implementation down enough so that we can start taking it apart more easily if required to add the needed features. On Tue, 2003-07-08 at 23:00, Scott M Stark wrote: And what interaction has there been with Nathan who originally responded to the rewrite query? -- /** * Hiram Chirino * Partner * Core Developers Network **/ --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite
On Tue, 2003-07-08 at 23:41, Nathan Phelps wrote: In line... Hiram, As you may know, we are going in a different direction with JMS than the original architecture coded by Norbert Lataille. We are doing a rewrite I guess I had it good. Norbert made a good start. At least basic pub/sub worked. That's better than starting from scratch. so patches to the old JBossMQ have a limited lifetime. That means that changes made to the old JBossMQ will most likely not be part of HEAD or the distribution as we move forward. You may be right. or wrong. The current JMS will ship until there is a better replacement. Do you plan to replace the current implementation with the peer based implementation you have been working on? Regards, Hiram Thanks, Nathan Phelps JMS Lead JBoss Group, L.L.C. -Original Message- From: [EMAIL PROTECTED] [mailto:jboss- [EMAIL PROTECTED] On Behalf Of Hiram Chirino Sent: Tuesday, July 08, 2003 8:41 PM To: [EMAIL PROTECTED] Subject: c/s JBossMQ status, was: [JBoss-dev] JBossMQ rewrite Hi guys, Over the past two weeks I have started to make a few improvements the current JBossMQ implementation that is in CVS HEAD. I would consider a large porting of what I did refactoring to simplify the current code base to allow future growth without having to sacrifice current features or performance. I just wanted to give provide an update on the development status of the original JBossMQ client-server JMS implementation. Here's a summary of what has changed: - It exclusively uses the remoting subsystem for client server communications. The existing IL subsystem has been removed. The 'async' remoting protocol allows two way communications solely over client sockets (this should allow applet/firewalled clients to connect to the server). Multiple JMS connections made by the client will also share a singe pool of sockets to communicate with the server. - Manually creating JBossMQ ConnectionFactory objects have been simplified since a remoting URI takes care setting up most of the connection options. - The server interceptors have been simplified since they now operate on a typeless invocations (it now matches the style of the other jboss interceptors) - The implementations for the p2p and pub/sub JMS API classes have been consolidated since that is the direction that the JMS 1.1 API is taking (being able to mix the us of p2p and pub/sub with in the same session). - Many (almost all) of the bug fixes/performance enhancements made in the 3.2 branch have been ported forward. - All the PMs except JDBC2 have been removed to make it easier to more quickly make enhancements to the persistence manager interface. I hope a cmp/jdo implementation can be created in the future. - the old org.jboss.mq.xml package that provided a simple xml dom was removed and all code that was using was changed to use dom4j instead. - JNDI Reference object handling was simplified (only StringRefs are used). - The sources were refactored so that the back-end messing server is much less coupled the javax.jms.* classes. The server side is still importing many of the JMSException classes, this can be cleaned up in the future. This might start to lead the way to allow this messaging server to support other messaging APIs (JavaSpaces?) Things to watch out for with this new version: - The jboss API to manually create ConnectionFactory and Destination objects has changed. Most users should not be affected since the JMS spec states that they should be using JNDI to get those objects. - Previous JBossMQ clients will not be able to connect to this new server version (since it is using remoting for communications). Those clients need to upgrade their jbossmq-client.jar. - The message format has changed so the new server will not be able to restore messages saved to persistence storage by the new server. I hope to commit the changes CVS in the next few days. I have 3 jms test cases that I still need to fix and then I still need to run the MDB test cases. Regards, Hiram -- /** * Hiram Chirino * Partner * Core Developers Network **/ --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This SF.Net email sponsored by: Parasoft Error proof Web apps, automate testing more. Download eval WebKing and get a free book. www.parasoft.com/bulletproofapps
RE: Antwort: RE: [JBoss-dev] JBossMQ rewrite
Sorry but the reloaded brand is already used by another JBoss project managed by Thomas Aardal to allow old JBoss client jars to be used with newer JBoss releases ;) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: lundi, 16. juin 2003 21:08 To: [EMAIL PROTECTED] Subject: RE: Antwort: RE: [JBoss-dev] JBossMQ rewrite As Bela and I have recently discussed with Tom Elrod, we all think that exposing JavaGroups as a transport of the remoting framework is indeed the best approach. However, I struggle with how the client-server nature of the remoting framework maps to the client-client nature of mulicast. I think this can indeed be overcome. However, as Tom doesn't have much time to spend on it, I think the road I'm gong to go down is to code directly to JavaGroups right now, and abstract it later. However, the real important point here FOR EVERYONE TO GET is that the multicast implementation is just ONE implementation of the client-side JMS interfaces. We will also support the traditional client-server based implementation that the new code is building toward. The only reason I'm even talking about the multicast stuff is because Bela showed me just how JavaGroups makes it so simple to implement and it will give us an opportunity to get something out the door in short order. Additionally, multicast will always provide better performance for pub/sub then a traditional client-server design. So for clients that want in-firewall messaging that is super fast, the multicast stuff will best serve their needs. For clients that need Internet messaging, etc. the traditional design will be used. We are VERY focused on building a best-of-breed JMS implementation. The current code causes us pain and that is all the motivation we need. I was very encouraged to talk to the guys last week and gather ideas. Andrian Brock is brilliant and has all sorts of ideas for the new JMS codebase that have all grown out of his dealings with the old codebase. We will not repeat the mistakes of the old codebase which is why we're starting anew from scratch. I'm going to get the project page on the website set up to better communicate the plan (which is something I should have done long ago) over the next week. Oh, and to address Marc's comment on the reloaded thing... that is just a code name for now--it sounds better then saying rewrite. The branding is clearly JMS/JBoss. Thanks, Nathan Phelps JMS/JBoss Project Lead JBoss Group, L.L.C. Quoting Bill Burke [EMAIL PROTECTED]: Nathan's design will not be based on JavaGroups, but will rather use JavaGroups as one type of transport mechanism. I would rather see JBoss Remoting used as an abstraction with JavaGroups as a plugin rather than JavaGroups at the center of things. BUTdon't let this requirement hold you up from getting a first iteration in place. Bill -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Bela Ban Sent: Monday, June 16, 2003 1:12 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: RE: [JBoss-dev] JBossMQ rewrite Hi Ulf, (2) message redelivery / message throttling clustering / failover since Nathan's design is based on JavaGroups, these issues are JavaGroups issues: - Message retransmission is handled by JavaGroups. - Failover: what do you understand by failover ? - Throttling: we are working on a multicast flow control protocol, JG currently ships with one, but it has a number of bugs and needs further work. I'm also working on a new flow control protocol. Also, note that you can run JG with TCP as transport, then you essentially have the classic JMS client-server implementation. (3) messaging system monitoring / administration I there a way to participate in the your ongoing rewrite ? Give us some more time; Nathan essentially designed the serverless JMS last weekend... -- Bela Ban http://www.javagroups.com Cell: (408) 316-4459 --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https
Antwort: RE: [JBoss-dev] JBossMQ rewrite
Hi Nathan, thanks for the information. I already have seen your first CVS code. Is there any further information available about the planed design and feature set ? You already gave some interesting insights about the p2p feature. When using jms in an enterprise production environment - as we would like to do - the following aspects are of even more importance ( and most of these isues are not handled very well in the current JBossMQ implementation ) (1) EFFICIENT handling of large / high numbers of PERSISTENT topic and queue messages (2) message redelivery / message throttling clustering / failover (3) messaging system monitoring / administration I there a way to participate in the your ongoing rewrite ? Regards Ulf Nathan Phelps [EMAIL PROTECTED] Gesendet von: [EMAIL PROTECTED] 15.06.2003 00:00 Bitte antworten an jboss-development An:[EMAIL PROTECTED] Kopie: Thema:RE: [JBoss-dev] JBossMQ rewrite JBossMQthe current code basewill continue to ship with JBoss 3.2 which is, and will remain for some time, the production version. Therefore, making changes to the current code base IS NOT worthless. However, I am working on a brand new implementation with assistance from Adrian, Bela, Bill, Tom Elrod, and Troy Daley. The framework code has recently been checked in to the jboss-jms module in CVS. It is early, but a start. In addition to the traditional client-server oriented JMS we're working on, at Bela's suggestion, I was able to implement a pure p2p implementation of the JMS topic messaging domain that does non-durable subscribers over JavaGroups. At Bill's request, we're going to get this code out there quickly (July). To my knowledge this will be the first pure p2p (server-less) JMS implementation in open source and it will provide very fast in-firewall publish and subscribe over multicast. Thanks, Nathan Phelps JMS/JBoss (Reloaded) Project Lead JBoss Group, L.L.C. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Friday, June 13, 2003 4:45 AM To: [EMAIL PROTECTED] Subject: [JBoss-dev] JBossMQ rewrite Can anyone give me some informations about the current state of the announced rewrite of JBossMQ for JBoss 4 ? Does it still make sense to implement needed features on the current JBossMQ implementation ? I don't want to spend time on something that gets nuked in short time :-) Regards Ulf --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: Antwort: RE: [JBoss-dev] JBossMQ rewrite
Hi Ulf, (2) message redelivery / message throttling clustering / failover since Nathan's design is based on JavaGroups, these issues are JavaGroups issues: - Message retransmission is handled by JavaGroups. - Failover: what do you understand by failover ? - Throttling: we are working on a multicast flow control protocol, JG currently ships with one, but it has a number of bugs and needs further work. I'm also working on a new flow control protocol. Also, note that you can run JG with TCP as transport, then you essentially have the classic JMS client-server implementation. (3) messaging system monitoring / administration I there a way to participate in the your ongoing rewrite ? Give us some more time; Nathan essentially designed the serverless JMS last weekend... -- Bela Ban http://www.javagroups.com Cell: (408) 316-4459 --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: Antwort: RE: [JBoss-dev] JBossMQ rewrite
Ulf, Our primary goal with JMS reloaded is to address the items you highlighted. Adrian and I had a very interesting discussion about number one, and Bill and I discussed clustering a bit in SF last week. So we are certainly focused on number 1 and 2. Number 3 is less of a concern to me right now. Overall, however, our goals are certainly aligned. I'm going to be putting together a formilized project plan next week that I'll publish on the website. From it, you and I should be able to determine where you have the time and skill to contriute the most. Thanks, Nathan Phelps JMS/JBoss (Reloaded) Project Lead JBoss Group, L.L.C. Quoting [EMAIL PROTECTED]: Hi Nathan, thanks for the information. I already have seen your first CVS code. Is there any further information available about the planed design and feature set ? You already gave some interesting insights about the p2p feature. When using jms in an enterprise production environment - as we would like to do - the following aspects are of even more importance ( and most of these isues are not handled very well in the current JBossMQ implementation ) (1) EFFICIENT handling of large / high numbers of PERSISTENT topic and queue messages (2) message redelivery / message throttling clustering / failover (3) messaging system monitoring / administration I there a way to participate in the your ongoing rewrite ? Regards Ulf Nathan Phelps [EMAIL PROTECTED] Gesendet von: [EMAIL PROTECTED] 15.06.2003 00:00 Bitte antworten an jboss-development An: [EMAIL PROTECTED] Kopie: Thema: RE: [JBoss-dev] JBossMQ rewrite JBossMQ?the current code base?will continue to ship with JBoss 3.2 which is, and will remain for some time, the production version. Therefore, making changes to the current code base IS NOT worthless. However, I am working on a brand new implementation with assistance from Adrian, Bela, Bill, Tom Elrod, and Troy Daley. The framework code has recently been checked in to the jboss-jms module in CVS. It is early, but a start. In addition to the traditional client-server oriented JMS we're working on, at Bela's suggestion, I was able to implement a pure p2p implementation of the JMS topic messaging domain that does non-durable subscribers over JavaGroups. At Bill's request, we're going to get this code out there quickly (July). To my knowledge this will be the first pure p2p (server-less) JMS implementation in open source and it will provide very fast in-firewall publish and subscribe over multicast. Thanks, Nathan Phelps JMS/JBoss (Reloaded) Project Lead JBoss Group, L.L.C. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Friday, June 13, 2003 4:45 AM To: [EMAIL PROTECTED] Subject: [JBoss-dev] JBossMQ rewrite Can anyone give me some informations about the current state of the announced rewrite of JBossMQ for JBoss 4 ? Does it still make sense to implement needed features on the current JBossMQ implementation ? I don't want to spend time on something that gets nuked in short time :-) Regards Ulf --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] JBossMQ rewrite
On the marketing front... Reloaded for the matrix is a great idea but will have too short a shelflife :) plus what we are doing is truly revolutionary so... Start hooking your wagon on that marketing speed train, marcf -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Nathan Phelps Sent: Saturday, June 14, 2003 6:00 PM To: [EMAIL PROTECTED] Subject: RE: [JBoss-dev] JBossMQ rewrite JBossMQthe current code basewill continue to ship with JBoss 3.2 which is, and will remain for some time, the production version. Therefore, making changes to the current code base IS NOT worthless. However, I am working on a brand new implementation with assistance from Adrian, Bela, Bill, Tom Elrod, and Troy Daley. The framework code has recently been checked in to the jboss-jms module in CVS. It is early, but a start. In addition to the traditional client-server oriented JMS we're working on, at Bela's suggestion, I was able to implement a pure p2p implementation of the JMS topic messaging domain that does non-durable subscribers over JavaGroups. At Bill's request, we're going to get this code out there quickly (July). To my knowledge this will be the first pure p2p (server-less) JMS implementation in open source and it will provide very fast in-firewall publish and subscribe over multicast. Thanks, Nathan Phelps JMS/JBoss (Reloaded) Project Lead JBoss Group, L.L.C. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Friday, June 13, 2003 4:45 AM To: [EMAIL PROTECTED] Subject: [JBoss-dev] JBossMQ rewrite Can anyone give me some informations about the current state of the announced rewrite of JBossMQ for JBoss 4 ? Does it still make sense to implement needed features on the current JBossMQ implementation ? I don't want to spend time on something that gets nuked in short time :-) Regards Ulf --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697- 6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: Antwort: RE: [JBoss-dev] JBossMQ rewrite
Nathan's design will not be based on JavaGroups, but will rather use JavaGroups as one type of transport mechanism. I would rather see JBoss Remoting used as an abstraction with JavaGroups as a plugin rather than JavaGroups at the center of things. BUTdon't let this requirement hold you up from getting a first iteration in place. Bill -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Bela Ban Sent: Monday, June 16, 2003 1:12 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: RE: [JBoss-dev] JBossMQ rewrite Hi Ulf, (2) message redelivery / message throttling clustering / failover since Nathan's design is based on JavaGroups, these issues are JavaGroups issues: - Message retransmission is handled by JavaGroups. - Failover: what do you understand by failover ? - Throttling: we are working on a multicast flow control protocol, JG currently ships with one, but it has a number of bugs and needs further work. I'm also working on a new flow control protocol. Also, note that you can run JG with TCP as transport, then you essentially have the classic JMS client-server implementation. (3) messaging system monitoring / administration I there a way to participate in the your ongoing rewrite ? Give us some more time; Nathan essentially designed the serverless JMS last weekend... -- Bela Ban http://www.javagroups.com Cell: (408) 316-4459 --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: Antwort: RE: [JBoss-dev] JBossMQ rewrite
Bill Burke wrote: Nathan's design will not be based on JavaGroups, but will rather use JavaGroups as one type of transport mechanism. Don't get all nervous; that's what I meant. 2 transports, one is the traditional c/s, the other one is implemented using JavaGroups. I would rather see JBoss Remoting used as an abstraction with JavaGroups as a plugin rather than JavaGroups at the center of things. Tom, Jeff, Nathan and I have tentatively scheduled JBoss bootcamp in Atlanta to be the place and time for an in-depth discussion of whether Remoting can accommodate a one-to-many paradigm in addition to a one-to-one paradigm. Nathan already has a thin abstraction layer over the transport, for both traditional and serverless design. -- Bela Ban http://www.javagroups.com Cell: (408) 316-4459 --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: Antwort: RE: [JBoss-dev] JBossMQ rewrite
I'm pretty excited about the JavaGroups implementation. Should scale quite well. I also really liked the idea of the AppServer being just another subscriber for durable topics. Should be interesting how all this develops over the next few months. Bill -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Bela Ban Sent: Monday, June 16, 2003 2:28 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: RE: [JBoss-dev] JBossMQ rewrite Bill Burke wrote: Nathan's design will not be based on JavaGroups, but will rather use JavaGroups as one type of transport mechanism. Don't get all nervous; that's what I meant. 2 transports, one is the traditional c/s, the other one is implemented using JavaGroups. I would rather see JBoss Remoting used as an abstraction with JavaGroups as a plugin rather than JavaGroups at the center of things. Tom, Jeff, Nathan and I have tentatively scheduled JBoss bootcamp in Atlanta to be the place and time for an in-depth discussion of whether Remoting can accommodate a one-to-many paradigm in addition to a one-to-one paradigm. Nathan already has a thin abstraction layer over the transport, for both traditional and serverless design. -- Bela Ban http://www.javagroups.com Cell: (408) 316-4459 --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: Antwort: RE: [JBoss-dev] JBossMQ rewrite
As Bela and I have recently discussed with Tom Elrod, we all think that exposing JavaGroups as a transport of the remoting framework is indeed the best approach. However, I struggle with how the client-server nature of the remoting framework maps to the client-client nature of mulicast. I think this can indeed be overcome. However, as Tom doesn't have much time to spend on it, I think the road I'm gong to go down is to code directly to JavaGroups right now, and abstract it later. However, the real important point here FOR EVERYONE TO GET is that the multicast implementation is just ONE implementation of the client-side JMS interfaces. We will also support the traditional client-server based implementation that the new code is building toward. The only reason I'm even talking about the multicast stuff is because Bela showed me just how JavaGroups makes it so simple to implement and it will give us an opportunity to get something out the door in short order. Additionally, multicast will always provide better performance for pub/sub then a traditional client-server design. So for clients that want in-firewall messaging that is super fast, the multicast stuff will best serve their needs. For clients that need Internet messaging, etc. the traditional design will be used. We are VERY focused on building a best-of-breed JMS implementation. The current code causes us pain and that is all the motivation we need. I was very encouraged to talk to the guys last week and gather ideas. Andrian Brock is brilliant and has all sorts of ideas for the new JMS codebase that have all grown out of his dealings with the old codebase. We will not repeat the mistakes of the old codebase which is why we're starting anew from scratch. I'm going to get the project page on the website set up to better communicate the plan (which is something I should have done long ago) over the next week. Oh, and to address Marc's comment on the reloaded thing... that is just a code name for now--it sounds better then saying rewrite. The branding is clearly JMS/JBoss. Thanks, Nathan Phelps JMS/JBoss Project Lead JBoss Group, L.L.C. Quoting Bill Burke [EMAIL PROTECTED]: Nathan's design will not be based on JavaGroups, but will rather use JavaGroups as one type of transport mechanism. I would rather see JBoss Remoting used as an abstraction with JavaGroups as a plugin rather than JavaGroups at the center of things. BUTdon't let this requirement hold you up from getting a first iteration in place. Bill -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Bela Ban Sent: Monday, June 16, 2003 1:12 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: RE: [JBoss-dev] JBossMQ rewrite Hi Ulf, (2) message redelivery / message throttling clustering / failover since Nathan's design is based on JavaGroups, these issues are JavaGroups issues: - Message retransmission is handled by JavaGroups. - Failover: what do you understand by failover ? - Throttling: we are working on a multicast flow control protocol, JG currently ships with one, but it has a number of bugs and needs further work. I'm also working on a new flow control protocol. Also, note that you can run JG with TCP as transport, then you essentially have the classic JMS client-server implementation. (3) messaging system monitoring / administration I there a way to participate in the your ongoing rewrite ? Give us some more time; Nathan essentially designed the serverless JMS last weekend... -- Bela Ban http://www.javagroups.com Cell: (408) 316-4459 --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] JBossMQ rewrite
Can anyone give me some informations about the current state of the announced rewrite of JBossMQ for JBoss 4 ? Does it still make sense to implement needed features on the current JBossMQ implementation ? I don't want to spend time on something that gets nuked in short time :-) Regards Ulf
RE: [JBoss-dev] JBossMQ rewrite
JBossMQthe current code basewill continue to ship with JBoss 3.2 which is, and will remain for some time, the production version. Therefore, making changes to the current code base IS NOT worthless. However, I am working on a brand new implementation with assistance from Adrian, Bela, Bill, Tom Elrod, and Troy Daley. The framework code has recently been checked in to the jboss-jms module in CVS. It is early, but a start. In addition to the traditional client-server oriented JMS we're working on, at Bela's suggestion, I was able to implement a pure p2p implementation of the JMS topic messaging domain that does non-durable subscribers over JavaGroups. At Bill's request, we're going to get this code out there quickly (July). To my knowledge this will be the first pure p2p (server-less) JMS implementation in open source and it will provide very fast in-firewall publish and subscribe over multicast. Thanks, Nathan Phelps JMS/JBoss (Reloaded) Project Lead JBoss Group, L.L.C. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Friday, June 13, 2003 4:45 AM To: [EMAIL PROTECTED] Subject: [JBoss-dev] JBossMQ rewrite Can anyone give me some informations about the current state of the announced rewrite of JBossMQ for JBoss 4 ? Does it still make sense to implement needed features on the current JBossMQ implementation ? I don't want to spend time on something that gets nuked in short time :-) Regards Ulf --- This SF.NET email is sponsored by: eBay Great deals on office technology -- on eBay now! Click here: http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] JBOSSMQ - JMSSecurityException
Hi all,i'm implementing a system with many clients and a server.This clients receives messages thought JMS. I'm trying that if I disconnect the net cable and after a while i reconnect the cable i was able to receipt the messages. I have a Thread that checks the connection state and reconnects the subscribers. The problem its that the socket beetween the server and my client it´s yet stablished now, when i want to re-establish the connection, jboss tells me that this client its allready connected to the server: [INFO,UILServerILService] Client request resulted in a server exception: javax.jms.JMSSecurityException: The login id has an assigned client id. That client id is already connected to the server!at org.jboss.mq.server.StateManager.checkUser(StateManager.java:195)at org.jboss.mq.server.JMSServer.checkUser(JMSServer.java:540)at org.jboss.mq.il.uil.UILServerILService.run(UILServerILService.java:286)at java.lang.Thread.run(Thread.java:479)In the clientside, the log is:An exception(JMSException) occured: org.jboss.mq.SpyJMSException: Cannot get a client IDorg.jboss.mq.SpyJMSException: Cannot get a client IDat org.jboss.mq.Connection.askForAnID(Connection.java:488)at org.jboss.mq.Connection.init(Connection.java:118)at org.jboss.mq.SpyConnection.init(SpyConnection.java:47)at org.jboss.mq.SpyConnectionFactory.createTopicConnection(SpyConnectionFactory.java:76)at com.ips.mainoutlet.messages.LocalACKMessageSubscriber.init(LocalACKMessageSubscriber.java:92)at com.ips.mainoutlet.messages.DataBaseUpdater.start(DataBaseUpdater.java:77)at com.ips.mainoutlet.tpv.application.proto_main_outlet.Frame1.setConnected(Frame1.java:2098)at com.ips.mainoutlet.bbdd_api.threadConexion.run(threadConexion.java:79)linked exception is:javax.jms.JMSSecurityException: The login id has an assigned client id. That client id is already connected to the server!linked exception is:javax.jms.JMSSecurityException: The login id has an assigned client id. That client id is already connected to the server! this exception occurs in the line: topicConnection=topicFactory.createTopicConnection(strIdTienda,strIdTienda); when i'm trying to reconnect. In other case, if i close my application, the sockets were closed, and if after an hour i want to reconnect, the messages are well receipt. How can I disconnect a subscriber? How can i break manually break the socket ? Any idea will be wellcomed.thanks a lot and sorry for my English, Sergio.
RE: [JBoss-dev] JBossMQ
that sounds right..
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]On Behalf Of John
Fawcett
Sent: Tuesday, January 14, 2003 8:14 PM
To: [EMAIL PROTECTED]
Subject: RE: [JBoss-dev] JBossMQ
Hi,
I want to make sure I understand the asynchronous delivery mechanism.
I've implemented my MessageConsumer to do the following:
Add self to Connection's message consumer list
While(consumer is open){
while(server is delivering synchronously){
Send Receive Requests until the Server is Drained
}
Wait for Asynch Delivery
}
Is this the proper pattern?
Thanks,
fawce
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of
Hiram Chirino
Sent: Friday, January 10, 2003 6:50 PM
To: [EMAIL PROTECTED]
Subject: RE: [JBoss-dev] JBossMQ
Your kinda right. the loop is there for the case where the destination
is queue based (p2p and durable subs). The polling happens when the
queue is full. Now when the queue is not full (or in the pub sub case,
there is no queue), then the thread goes into asynch mode and it waits
for the message to get delivered async via the ClientIL.receive method.
I'll comment the code a little:
// gets the next message in queue or registers us for asynch
delivery if none available.
mes = session.connection.receive( subscription, 0 );
if ( mes == null ) // should always be null for pub-sub case.
{
// start waiting for the message to get delivered asynch
waitingForMessage = true;
while ( ( messages.isEmpty() !closed ) || (
!session.running ) )
{
try
{
// messages gets signaled once ClientIL.receive finishes
processing the message.
messages.wait();
} catch ( InterruptedException e )
{
}
}
if ( closed )
{
waitingForMessage = false;
break outer;
}
// the message sent via ClientIL.receive should now be sitting
in messages
mes = ( SpyMessage )messages.removeFirst();
waitingForMessage = false;
}
I hope that helped! I think the XIL is great idea! We might even be
able to develop a c base API to access mq services (important in the
integration space).
Regards,
Hiram
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]On Behalf Of
John Fawcett
Sent: Thursday, January 09, 2003 8:09 PM
To: [EMAIL PROTECTED]
Subject: [JBoss-dev] JBossMQ
Hi,
I am working on a new invocation layer (IL) for JbossMQ that encodes
all communication in Xml (XIL). I've got the IL pretty near
completion, and I am working on a C# jbossmq client. I am trying to
develop the TopicSubsciber, which is an extension of MessageConsumer.
In reviewing the code in the jbossmq java sources, it looks to me like
the client to a topic actually runs a loop sending receive requests to
the server regularly.
Is this really necessary? Once the connection has been established,
why can't the server just invoke the ClientIL.receive method (which
actually sends the message to the client) when a message arrives at
the destination? It looks to me like the current implementation is not
truly asynchronous...
What am I missing?
Thanks,
fawce
-
John Fawcett
CTO, Tamale Software, LLC
26 Fox Road
Waltham, MA 02451
---
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
___
Jboss-development mailing list [EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
---
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
___
Jboss-development mailing list [EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
---
This SF.NET email is sponsored by: Take your first step towards giving
your online business a competitive advantage. Test-drive a Thawte SSL
certificate - our easy online guide will show you how. Click here to get
started: http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0027en
___
Jboss-development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
---
This SF.NET email is sponsored by: Take your first step towards giving
your online
RE: [JBoss-dev] JBossMQ
Hi,
I want to make sure I understand the asynchronous delivery mechanism.
I've implemented my MessageConsumer to do the following:
Add self to Connection's message consumer list
While(consumer is open){
while(server is delivering synchronously){
Send Receive Requests until the Server is Drained
}
Wait for Asynch Delivery
}
Is this the proper pattern?
Thanks,
fawce
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of
Hiram Chirino
Sent: Friday, January 10, 2003 6:50 PM
To: [EMAIL PROTECTED]
Subject: RE: [JBoss-dev] JBossMQ
Your kinda right. the loop is there for the case where the destination
is queue based (p2p and durable subs). The polling happens when the
queue is full. Now when the queue is not full (or in the pub sub case,
there is no queue), then the thread goes into asynch mode and it waits
for the message to get delivered async via the ClientIL.receive method.
I'll comment the code a little:
// gets the next message in queue or registers us for asynch
delivery if none available.
mes = session.connection.receive( subscription, 0 );
if ( mes == null ) // should always be null for pub-sub case.
{
// start waiting for the message to get delivered asynch
waitingForMessage = true;
while ( ( messages.isEmpty() !closed ) || (
!session.running ) )
{
try
{
// messages gets signaled once ClientIL.receive finishes
processing the message.
messages.wait();
} catch ( InterruptedException e )
{
}
}
if ( closed )
{
waitingForMessage = false;
break outer;
}
// the message sent via ClientIL.receive should now be sitting
in messages
mes = ( SpyMessage )messages.removeFirst();
waitingForMessage = false;
}
I hope that helped! I think the XIL is great idea! We might even be
able to develop a c base API to access mq services (important in the
integration space).
Regards,
Hiram
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]On Behalf Of
John Fawcett
Sent: Thursday, January 09, 2003 8:09 PM
To: [EMAIL PROTECTED]
Subject: [JBoss-dev] JBossMQ
Hi,
I am working on a new invocation layer (IL) for JbossMQ that encodes
all communication in Xml (XIL). I've got the IL pretty near
completion, and I am working on a C# jbossmq client. I am trying to
develop the TopicSubsciber, which is an extension of MessageConsumer.
In reviewing the code in the jbossmq java sources, it looks to me like
the client to a topic actually runs a loop sending receive requests to
the server regularly.
Is this really necessary? Once the connection has been established,
why can't the server just invoke the ClientIL.receive method (which
actually sends the message to the client) when a message arrives at
the destination? It looks to me like the current implementation is not
truly asynchronous...
What am I missing?
Thanks,
fawce
-
John Fawcett
CTO, Tamale Software, LLC
26 Fox Road
Waltham, MA 02451
---
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
___
Jboss-development mailing list [EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
---
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
___
Jboss-development mailing list [EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
---
This SF.NET email is sponsored by: Take your first step towards giving
your online business a competitive advantage. Test-drive a Thawte SSL
certificate - our easy online guide will show you how. Click here to get
started: http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0027en
___
Jboss-development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] JBossMQ, multiplexor test
Hi, I've isolated the hanging behavior to the writer threads in multiplexor test. I re-wrote the tests to run over a socket and use a client and a server. When I run the test with the server configured to: Write on channel 1 Write on channel 2 Read from channel 1 And the client configured to: Read from channel 1 Read from channel 2 Write to channel 1 The Server hangs part-way (lock up happens at an unpredictable time) through the writing iterations. I am running all of this on WinXP pro, with Jdk1.4.1 ... Anyone have any ideas? Thanks, fawce -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of John Fawcett Sent: Sunday, January 12, 2003 6:50 PM To: [EMAIL PROTECTED] Subject: RE: [JBoss-dev] JBossMQ, multiplexor test Thanks for the explanation of the receive loop Hiram. I am trying to run the Multiplexor test included in org.jboss.mq.il.uil.multiplexor. For some indiscernible reason, the test is freezing after starting the first stream. I am trying to run the test under jdk1.4.1_01 -- are there any known compatibility problems? Or can someone verify that the test runs ? Thanks, fawce --- This SF.NET email is sponsored by: FREE SSL Guide from Thawte are you planning your Web Server Security? Click here to get a FREE Thawte SSL guide and find the answers to all your SSL security issues. http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0026en ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] JBossMQ, multiplexor test
Thanks for the explanation of the receive loop Hiram.
I am trying to run the Multiplexor test included in
org.jboss.mq.il.uil.multiplexor. For some indiscernible reason, the test
is freezing after starting the first stream.
I am trying to run the test under jdk1.4.1_01 -- are there any known
compatibility problems? Or can someone verify that the test runs ?
Thanks,
fawce
-
John Fawcett
CTO, Tamale Software, LLC
26 Fox Road
Waltham, MA 02451
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of
Hiram Chirino
Sent: Friday, January 10, 2003 6:50 PM
To: [EMAIL PROTECTED]
Subject: RE: [JBoss-dev] JBossMQ
Your kinda right. the loop is there for the case where the destination
is
queue based (p2p and durable subs). The polling happens when the queue
is
full. Now when the queue is not full (or in the pub sub case, there is
no
queue), then the thread goes into asynch mode and it waits for the
message
to get delivered async via the ClientIL.receive method. I'll comment
the
code a little:
// gets the next message in queue or registers us for asynch
delivery
if none available.
mes = session.connection.receive( subscription, 0 );
if ( mes == null ) // should always be null for pub-sub case.
{
// start waiting for the message to get delivered asynch
waitingForMessage = true;
while ( ( messages.isEmpty() !closed ) || (
!session.running ) )
{
try
{
// messages gets signaled once ClientIL.receive finishes
processing the message.
messages.wait();
} catch ( InterruptedException e )
{
}
}
if ( closed )
{
waitingForMessage = false;
break outer;
}
// the message sent via ClientIL.receive should now be sitting
in
messages
mes = ( SpyMessage )messages.removeFirst();
waitingForMessage = false;
}
I hope that helped! I think the XIL is great idea! We might even be
able
to develop a c base API to access mq services (important in the
integration
space).
Regards,
Hiram
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]On Behalf Of
John
Fawcett
Sent: Thursday, January 09, 2003 8:09 PM
To: [EMAIL PROTECTED]
Subject: [JBoss-dev] JBossMQ
Hi,
I am working on a new invocation layer (IL) for JbossMQ that encodes
all
communication in Xml (XIL). I've got the IL pretty near completion,
and
I am working on a C# jbossmq client. I am trying to develop the
TopicSubsciber, which is an extension of MessageConsumer. In reviewing
the code in the jbossmq java sources, it looks to me like the client
to
a topic actually runs a loop sending receive requests to the server
regularly.
Is this really necessary? Once the connection has been established,
why
can't the server just invoke the ClientIL.receive method (which
actually
sends the message to the client) when a message arrives at the
destination?
It looks to me like the current implementation is not truly
asynchronous...
What am I missing?
Thanks,
fawce
-
John Fawcett
CTO, Tamale Software, LLC
26 Fox Road
Waltham, MA 02451
---
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
___
Jboss-development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
---
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
___
Jboss-development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
---
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
___
Jboss-development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] JBossMQ
Your kinda right. the loop is there for the case where the destination is
queue based (p2p and durable subs). The polling happens when the queue is
full. Now when the queue is not full (or in the pub sub case, there is no
queue), then the thread goes into asynch mode and it waits for the message
to get delivered async via the ClientIL.receive method. I'll comment the
code a little:
// gets the next message in queue or registers us for asynch delivery
if none available.
mes = session.connection.receive( subscription, 0 );
if ( mes == null ) // should always be null for pub-sub case.
{
// start waiting for the message to get delivered asynch
waitingForMessage = true;
while ( ( messages.isEmpty() !closed ) || ( !session.running ) )
{
try
{
// messages gets signaled once ClientIL.receive finishes
processing the message.
messages.wait();
} catch ( InterruptedException e )
{
}
}
if ( closed )
{
waitingForMessage = false;
break outer;
}
// the message sent via ClientIL.receive should now be sitting in
messages
mes = ( SpyMessage )messages.removeFirst();
waitingForMessage = false;
}
I hope that helped! I think the XIL is great idea! We might even be able
to develop a c base API to access mq services (important in the integration
space).
Regards,
Hiram
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]On Behalf Of John
Fawcett
Sent: Thursday, January 09, 2003 8:09 PM
To: [EMAIL PROTECTED]
Subject: [JBoss-dev] JBossMQ
Hi,
I am working on a new invocation layer (IL) for JbossMQ that encodes all
communication in Xml (XIL). I've got the IL pretty near completion, and
I am working on a C# jbossmq client. I am trying to develop the
TopicSubsciber, which is an extension of MessageConsumer. In reviewing
the code in the jbossmq java sources, it looks to me like the client to
a topic actually runs a loop sending receive requests to the server
regularly.
Is this really necessary? Once the connection has been established, why
can't the server just invoke the ClientIL.receive method (which actually
sends the message to the client) when a message arrives at the
destination?
It looks to me like the current implementation is not truly
asynchronous...
What am I missing?
Thanks,
fawce
-
John Fawcett
CTO, Tamale Software, LLC
26 Fox Road
Waltham, MA 02451
---
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
___
Jboss-development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
---
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
___
Jboss-development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] JBossMQ
Hi, I am working on a new invocation layer (IL) for JbossMQ that encodes all communication in Xml (XIL). I've got the IL pretty near completion, and I am working on a C# jbossmq client. I am trying to develop the TopicSubsciber, which is an extension of MessageConsumer. In reviewing the code in the jbossmq java sources, it looks to me like the client to a topic actually runs a loop sending receive requests to the server regularly. Is this really necessary? Once the connection has been established, why can't the server just invoke the ClientIL.receive method (which actually sends the message to the client) when a message arrives at the destination? It looks to me like the current implementation is not truly asynchronous... What am I missing? Thanks, fawce - John Fawcett CTO, Tamale Software, LLC 26 Fox Road Waltham, MA 02451 --- This SF.NET email is sponsored by: SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See! http://www.vasoftware.com ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] jbossmq and jca 1.5
Hey David.. I want to see If I can start working on this. I know like amost zero about jca but I know quite a bit about the XA stuff and how jbossmq txs work with the MDB. anyways.. I guess what I need to know is how is the contianer going to initialize the jms stuff so that it can deliver the container it's messages. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of David Jencks Sent: Friday, August 30, 2002 1:35 PM To: jboss-dev Subject: [JBoss-dev] jbossmq and jca 1.5 I started looking at modifying jbossmq and/or the jmsra adapter to work with the j2ee connector architecture 1.5 facilities. I am definitely not a jms expert so a lot of what I say may not make sense. Here's what the new spec provides that I think is relevant: thread pooling through the WorkManager interface. You submit Work instances to be done in other threads. The app server controls the thread pooling. message inflow through the MessageEndpoint interfaces. In particular, I think we should use Option B which involves the jms system calling, on a MessageEndpoint supplied from the app server, beforeDelivery([the onMessage method]); //this starts the jta tx and informs the adapter via an XAResource the adapter supplied earlier. onMessage(message); //actual message delivery to MessageListener afterDelivery(); //ends the tx, again the adapter is informed via the XAResource. I keep getting lost looking at the jms code. My impression so far is that although the jca 1 jmsra adapter appears to work ok without modifying jbossmq, using the contracts mentioned above will require additional code in jbossmq itself, namely an additional way of delivering messages within a transaction. Does this make sense? Do any of the jbossmq experts want to work on this? There are very simple examples of using the work and message endpoint interfaces in the testsuite in .../jca/inflow. I haven't written the deployment portion of the connector 1.5 support yet: I'm hoping for a real adapter example that can be used to test it. However, I think for now everything needed can be set up without a deployer as explicit mbeans: this is what I did in the tests. Thanks david jencks --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE! https://www.inphonic.com/r.asp?r=sourceforge1refcode1=vs3390 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq and jca 1.5
Hey Hiram, Glad to see you're getting interested. Unfortunately I leave for the weekend in a few minutes... more on monday. The part of jca 1.5 I haven't written is the deployment of the adapter. If you get that far, you should be able to write a simple mbean to create your ResourceAdapter and any adapter specific objects you need, then I can make it generic. Your adapter should have a ResourceAdapter object that sets up the environment. It will be provided a BootstrapContext object from which it can get a WorkManager. The WorkManager is a thread pool with transaction import capabilities. You won't need the tx import capabilities, but you should use the thread pool. I plan for the deployer to supply a WorkManager per deployed adapter, configured according to a jboss-specific dd, similar to the connection pool configuration. Instead, use the Message Inflow stuff. There's a unit test showing how it works with transactions - org.jboss.test.jca.test.MessageEndpointUnitTestCase. The deployment system I'm thinking of is to deploy the ResourceAdapter and other adapter specific objects as xmbeans by generating xmbean xml deployment descriptors for them. BTW, I modified the Trunk invoker to make our tm distributed, using the jca transaction import facilities. It compiles and runs, but I haven't managed to set up a test environment for it with 2 servers yet. Thanks! david jencks On 2002.10.05 02:58:45 -0400 Hiram Chirino wrote: Hey David.. I want to see If I can start working on this. I know like amost zero about jca but I know quite a bit about the XA stuff and how jbossmq txs work with the MDB. anyways.. I guess what I need to know is how is the contianer going to initialize the jms stuff so that it can deliver the container it's messages. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of David Jencks Sent: Friday, August 30, 2002 1:35 PM To: jboss-dev Subject: [JBoss-dev] jbossmq and jca 1.5 I started looking at modifying jbossmq and/or the jmsra adapter to work with the j2ee connector architecture 1.5 facilities. I am definitely not a jms expert so a lot of what I say may not make sense. Here's what the new spec provides that I think is relevant: thread pooling through the WorkManager interface. You submit Work instances to be done in other threads. The app server controls the thread pooling. message inflow through the MessageEndpoint interfaces. In particular, I think we should use Option B which involves the jms system calling, on a MessageEndpoint supplied from the app server, beforeDelivery([the onMessage method]); //this starts the jta tx and informs the adapter via an XAResource the adapter supplied earlier. onMessage(message); //actual message delivery to MessageListener afterDelivery(); //ends the tx, again the adapter is informed via the XAResource. I keep getting lost looking at the jms code. My impression so far is that although the jca 1 jmsra adapter appears to work ok without modifying jbossmq, using the contracts mentioned above will require additional code in jbossmq itself, namely an additional way of delivering messages within a transaction. Does this make sense? Do any of the jbossmq experts want to work on this? There are very simple examples of using the work and message endpoint interfaces in the testsuite in .../jca/inflow. I haven't written the deployment portion of the connector 1.5 support yet: I'm hoping for a real adapter example that can be used to test it. However, I think for now everything needed can be set up without a deployer as explicit mbeans: this is what I did in the tests. Thanks david jencks --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE! https://www.inphonic.com/r.asp?r=sourceforge1refcode1=vs3390 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq and jca 1.5
Hi, I am sorry I have not had the time to do anything on the jca1.5 integration. I have not even had time to read the new spec. From what you say I would however draw the following conclusions: 1. The jca 1.5 have defined a new contract superceding chapter 8 in the JMS spec, which means that each JMS provider will have to roll its own JMS JCC provider adapter. 2. This only affecs asynchronous use, i.e only JMSContainerInvoker. 3. We would therefore perhaps want a jms.JCAContainerInvoker wich works agains the new JCA. 4. On the other hand we have been propagating for a year and a half now that chapter 8 in the JMS spec give us all we need, and we have integrated JBossMQ, SonicMQ and SwiftMQ through the chapter 8 api. 5. Adding a native RA to JBossMQ would entail recreating the Conumer stuff in the new RA. 6. My advice would actually be this: write a jca2chapter8 converter. To do this you would thake the stuff from StdServerSession, StdServerSessionPool and JMSContainerInvoker and integrate into the RA. The RA would continue to work against a ProviderAdapter and chapter 8, but would also be able to work with the new JCA. Look for example in StdServerSession - there you can split the TX logic and do your callback. The old way is that the appserver provides the thread pool (StdServerSessionPool), the JMS provider gets a (Std)ServerSession from the appserverpool, stuff its on session in it and let the appserver do its work. Seems to be verry similar. Hope this helps somewhat. //Peter On 30 Aug, David Jencks wrote: I started looking at modifying jbossmq and/or the jmsra adapter to work with the j2ee connector architecture 1.5 facilities. I am definitely not a jms expert so a lot of what I say may not make sense. Here's what the new spec provides that I think is relevant: thread pooling through the WorkManager interface. You submit Work instances to be done in other threads. The app server controls the thread pooling. message inflow through the MessageEndpoint interfaces. In particular, I think we should use Option B which involves the jms system calling, on a MessageEndpoint supplied from the app server, beforeDelivery([the onMessage method]); //this starts the jta tx and informs the adapter via an XAResource the adapter supplied earlier. onMessage(message); //actual message delivery to MessageListener afterDelivery(); //ends the tx, again the adapter is informed via the XAResource. I keep getting lost looking at the jms code. My impression so far is that although the jca 1 jmsra adapter appears to work ok without modifying jbossmq, using the contracts mentioned above will require additional code in jbossmq itself, namely an additional way of delivering messages within a transaction. Does this make sense? Do any of the jbossmq experts want to work on this? There are very simple examples of using the work and message endpoint interfaces in the testsuite in .../jca/inflow. I haven't written the deployment portion of the connector 1.5 support yet: I'm hoping for a real adapter example that can be used to test it. However, I think for now everything needed can be set up without a deployer as explicit mbeans: this is what I did in the tests. Thanks david jencks --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE! https://www.inphonic.com/r.asp?r=sourceforge1refcode1=vs3390 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development -- Peter AntmanChief Systems Architect, Business Development Technology in Media, Box 34105 100 26 Stockholm WWW: http://www.tim.se WWW: http://www.backsource.org Email: [EMAIL PROTECTED] Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE! https://www.inphonic.com/r.asp?r=sourceforge1refcode1=vs3390 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq and jca 1.5
Many thanks, peter. I have to work on some other things for a while also, I will try to look into this more. Your pointers will be a big help in seeing how it works. Thanks david jencks On 2002.09.03 03:25:26 -0400 Peter Antman wrote: Hi, I am sorry I have not had the time to do anything on the jca1.5 integration. I have not even had time to read the new spec. From what you say I would however draw the following conclusions: 1. The jca 1.5 have defined a new contract superceding chapter 8 in the JMS spec, which means that each JMS provider will have to roll its own JMS JCC provider adapter. 2. This only affecs asynchronous use, i.e only JMSContainerInvoker. 3. We would therefore perhaps want a jms.JCAContainerInvoker wich works agains the new JCA. 4. On the other hand we have been propagating for a year and a half now that chapter 8 in the JMS spec give us all we need, and we have integrated JBossMQ, SonicMQ and SwiftMQ through the chapter 8 api. 5. Adding a native RA to JBossMQ would entail recreating the Conumer stuff in the new RA. 6. My advice would actually be this: write a jca2chapter8 converter. To do this you would thake the stuff from StdServerSession, StdServerSessionPool and JMSContainerInvoker and integrate into the RA. The RA would continue to work against a ProviderAdapter and chapter 8, but would also be able to work with the new JCA. Look for example in StdServerSession - there you can split the TX logic and do your callback. The old way is that the appserver provides the thread pool (StdServerSessionPool), the JMS provider gets a (Std)ServerSession from the appserverpool, stuff its on session in it and let the appserver do its work. Seems to be verry similar. Hope this helps somewhat. //Peter On 30 Aug, David Jencks wrote: I started looking at modifying jbossmq and/or the jmsra adapter to work with the j2ee connector architecture 1.5 facilities. I am definitely not a jms expert so a lot of what I say may not make sense. Here's what the new spec provides that I think is relevant: thread pooling through the WorkManager interface. You submit Work instances to be done in other threads. The app server controls the thread pooling. message inflow through the MessageEndpoint interfaces. In particular, I think we should use Option B which involves the jms system calling, on a MessageEndpoint supplied from the app server, beforeDelivery([the onMessage method]); //this starts the jta tx and informs the adapter via an XAResource the adapter supplied earlier. onMessage(message); //actual message delivery to MessageListener afterDelivery(); //ends the tx, again the adapter is informed via the XAResource. I keep getting lost looking at the jms code. My impression so far is that although the jca 1 jmsra adapter appears to work ok without modifying jbossmq, using the contracts mentioned above will require additional code in jbossmq itself, namely an additional way of delivering messages within a transaction. Does this make sense? Do any of the jbossmq experts want to work on this? There are very simple examples of using the work and message endpoint interfaces in the testsuite in .../jca/inflow. I haven't written the deployment portion of the connector 1.5 support yet: I'm hoping for a real adapter example that can be used to test it. However, I think for now everything needed can be set up without a deployer as explicit mbeans: this is what I did in the tests. Thanks david jencks --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE! https://www.inphonic.com/r.asp?r=sourceforge1refcode1=vs3390 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development -- Peter Antman Chief Systems Architect, Business Development Technology in Media, Box 34105 100 26 Stockholm WWW: http://www.tim.seWWW: http://www.backsource.org Email: [EMAIL PROTECTED] Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE! https://www.inphonic.com/r.asp?r=sourceforge1refcode1=vs3390 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE!
[JBoss-dev] jbossmq and jca 1.5
I started looking at modifying jbossmq and/or the jmsra adapter to work with the j2ee connector architecture 1.5 facilities. I am definitely not a jms expert so a lot of what I say may not make sense. Here's what the new spec provides that I think is relevant: thread pooling through the WorkManager interface. You submit Work instances to be done in other threads. The app server controls the thread pooling. message inflow through the MessageEndpoint interfaces. In particular, I think we should use Option B which involves the jms system calling, on a MessageEndpoint supplied from the app server, beforeDelivery([the onMessage method]); //this starts the jta tx and informs the adapter via an XAResource the adapter supplied earlier. onMessage(message); //actual message delivery to MessageListener afterDelivery(); //ends the tx, again the adapter is informed via the XAResource. I keep getting lost looking at the jms code. My impression so far is that although the jca 1 jmsra adapter appears to work ok without modifying jbossmq, using the contracts mentioned above will require additional code in jbossmq itself, namely an additional way of delivering messages within a transaction. Does this make sense? Do any of the jbossmq experts want to work on this? There are very simple examples of using the work and message endpoint interfaces in the testsuite in .../jca/inflow. I haven't written the deployment portion of the connector 1.5 support yet: I'm hoping for a real adapter example that can be used to test it. However, I think for now everything needed can be set up without a deployer as explicit mbeans: this is what I did in the tests. Thanks david jencks --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE! https://www.inphonic.com/r.asp?r=sourceforge1refcode1=vs3390 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq and jca 1.5
Hi David I am still looking into the code to become an JMS expert but I would like to help to implement this. Because I added the Common Interfaces of JMS 1.1 it would like to use this opportunity to make the appropriate changes here. This week I am pretty busy. Have fun - Andy I started looking at modifying jbossmq and/or the jmsra adapter to work with the j2ee connector architecture 1.5 facilities. I am definitely not a jms expert so a lot of what I say may not make sense. Here's what the new spec provides that I think is relevant: thread pooling through the WorkManager interface. You submit Work instances to be done in other threads. The app server controls the thread pooling. message inflow through the MessageEndpoint interfaces. In particular, I think we should use Option B which involves the jms system calling, on a MessageEndpoint supplied from the app server, beforeDelivery([the onMessage method]); //this starts the jta tx and informs the adapter via an XAResource the adapter supplied earlier. onMessage(message); //actual message delivery to MessageListener afterDelivery(); //ends the tx, again the adapter is informed via the XAResource. I keep getting lost looking at the jms code. My impression so far is that although the jca 1 jmsra adapter appears to work ok without modifying jbossmq, using the contracts mentioned above will require additional code in jbossmq itself, namely an additional way of delivering messages within a transaction. Does this make sense? Do any of the jbossmq experts want to work on this? There are very simple examples of using the work and message endpoint interfaces in the testsuite in .../jca/inflow. I haven't written the deployment portion of the connector 1.5 support yet: I'm hoping for a real adapter example that can be used to test it. However, I think for now everything needed can be set up without a deployer as explicit mbeans: this is what I did in the tests. Thanks david jencks --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE! https://www.inphonic.com/r.asp?r=sourceforge1refcode1=vs3390 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE! https://www.inphonic.com/r.asp?r=sourceforge1refcode1=vs3390 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] JBossMQ... what happened?
Um, what is up with JBossMQ? I'm working of the latest CVS (actually the problem below has been happening for the past few weeks) and JMS is absent from JNDI. The sample Queues and Topics are being created because I see them in the Agent View under the jboss.mq.destination section. I also see all the invocation layers under the jboss.mq section. However, when I do a JNDIView, there is no queue/topic sub-contexts in the global JNDI namespace nor are there any connection factories. Is JMS disabled somehow in the default build? If so, why are the topics/queues being created if they aren't going to be made available? Thanks, Michael ___ Don't miss the 2002 Sprint PCS Application Developer's Conference August 25-28 in Las Vegas -- http://devcon.sprintpcs.com/adp/index.cfm ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] JBossMQ JMX Interface change
Hi guys, I just finished commiting some major refactoring of the JBossMQ server. It just involved formaizing the interceptor pattern that peter started with the security manager addition. I also refactored some of the JMX MBeans to thier own package. So this is your notice: THE JMX INTERFACE TO JBOSSMQ CHANGED. Ok, it didn't change that much but, it changed. I hope this does not break too many things. I'll be commiting these changes to Branch_3_0 soon. Regards, Hiram _ Chat with friends online, try MSN Messenger: http://messenger.msn.com ___ Have big pipes? SourceForge.net is looking for download mirrors. We supply the hardware. You get the recognition. Email Us: [EMAIL PROTECTED] ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ JavaCC Problem
It should be possible to define any sort of grammer with JavaCC. Perhaps the root production does not have the proper terminators to single the end of a selector? --jason Hiram Chirino wrote: To all JavaCC Gurus, The javacc grammar that JBossMQ is using has a problem. It can't detect that the selector: definitely not a message selector! is an invalid selector. It just parses the first itentifier, definitely and then skips over the rest of the selector. Is there an easy way for javacc to pick up that there are some excess tokens and spit out an error??? Regards, Hiram _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp. ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Have big pipes? SourceForge.net is looking for download mirrors. We supply the hardware. You get the recognition. Email Us: [EMAIL PROTECTED] Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ JavaCC Problem
I'm looking into it. Scott Stark Chief Technology Officer JBoss Group, LLC - Original Message - From: Hiram Chirino [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Monday, April 29, 2002 9:50 PM Subject: [JBoss-dev] JBossMQ JavaCC Problem To all JavaCC Gurus, The javacc grammar that JBossMQ is using has a problem. It can't detect that the selector: definitely not a message selector! is an invalid selector. It just parses the first itentifier, definitely and then skips over the rest of the selector. Is there an easy way for javacc to pick up that there are some excess tokens and spit out an error??? Regards, Hiram _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp. ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ JavaCC Problem
Fixed. All parser unit tests now run without errors. Scott Stark Chief Technology Officer JBoss Group, LLC - Original Message - From: Hiram Chirino [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Monday, April 29, 2002 9:50 PM Subject: [JBoss-dev] JBossMQ JavaCC Problem To all JavaCC Gurus, The javacc grammar that JBossMQ is using has a problem. It can't detect that the selector: definitely not a message selector! is an invalid selector. It just parses the first itentifier, definitely and then skips over the rest of the selector. Is there an easy way for javacc to pick up that there are some excess tokens and spit out an error???
[JBoss-dev] JBossMQ JavaCC Problem
To all JavaCC Gurus, The javacc grammar that JBossMQ is using has a problem. It can't detect that the selector: definitely not a message selector! is an invalid selector. It just parses the first itentifier, definitely and then skips over the rest of the selector. Is there an easy way for javacc to pick up that there are some excess tokens and spit out an error??? Regards, Hiram _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp. ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] JBossMQ Questions
Hi Hiram Just started to implement JSR-77 performance monitoring on JBossMQ (was just in the flow after the recent events). I came up with some questions but let me explain what JSR-77 spec. contains: a JMS stats contains: - list of connections - which contains a list of sesssions - which contains a list of consumers and producers I started adding JBoss MQ code in JMSServer because this seems to be the central component of JBossMQ. Now I need to find out what connections exists and retrieve informations from them inclusive the list of sessions and from this the list of sender/receivers or list of publisher/subscribers. All this seems not to be a problem. But I don't see the connnection between JMSServer and SpyConnection. Is there a way to get there inclusive going over JNDI or JMX ? Thanx x Andreas Schaefer Senior Consultant JBoss Group, LLC x ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] JBossMQ Problem with Topics
Hi Geeks When a message is sent to a non-durable, non-persistent topic and no subscriber are available is then the message kept in memory or dropped ? Thanx x Andreas Schaefer Senior Consultant JBoss Group, LLC x ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] JBossMQ Problem with Topics
Dropped, as per the JMS spec. -Original Message- From: Andreas Schaefer [mailto:[EMAIL PROTECTED]] Sent: Thursday, April 18, 2002 11:59 AM To: Hiram Chirino Cc: [EMAIL PROTECTED] Subject: [JBoss-dev] JBossMQ Problem with Topics Hi Geeks When a message is sent to a non-durable, non-persistent topic and no subscriber are available is then the message kept in memory or dropped ? Thanx x Andreas Schaefer Senior Consultant JBoss Group, LLC x ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development --- Incoming mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.350 / Virus Database: 196 - Release Date: 4/17/2002 --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.350 / Virus Database: 196 - Release Date: 4/17/2002 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] JBossMQ Memory Leak
Hi Hiram I just heared from a customer that they found a memory leak in JBossMQ with a profiler. He promised to send information about it soon I am also working on a long term test. Because of that I will create a marathon test case for JBossMQ in the next days to investigate this issue further. So I will let you know as soon as I get more information. Andy x Andreas Schaefer Senior Consultant JBoss Group, LLC x ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] jbossmq-oldstate.xml
Is this file really needed? --jason ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq-oldstate.xml
On 7 Mar, Jason Dillon wrote: It's there to easy transition. The old state manager is still around for people having tons of users in their old state format, so it's mostly there to document the format of the OldStatemanager since it do not have a DTD. Maybe it should not get copy'd over during the jboss server build. //Peter Is this file really needed? --jason ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development -- Peter AntmanChief Systems Architect, Business Development Technology in Media, Box 34105 100 26 Stockholm WWW: http://www.tim.se WWW: http://www.backsource.org Email: [EMAIL PROTECTED] Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ and JAAS
On 22 Feb, Scott M Stark wrote: [...] The new resource adaptor security will only be implemented in 3.x. The current 2.4 code base does not matter. Well, a connection hold in vm may possibly be used by different threads, but the connection should, in case it was started with userid/pwd, hold ontoit itself, and it ought to be the server side that does this. Instead of relying on a thread local as in SecurityAssociation and the security manager (for subject), we could perhaps store the subject in ClassContextLocal storage. But I am reallt not shure about this now. This will be an implementation detail of your resource adaptor. There will be no use of SecurityAssociation at the resource adaptor level. The current status for JMS is this regarding JCA: the is no support for asynchronous messaging. Ad to this that to use a JCA ra i a non managed environment is a pain in the neck. At least half of the users using JBossMQ does it on a pure/old client-server setup (no J2EE on the client side). This mean that we can not only provide JCA ra for JBossMQ, but must support the vanilla JMS, and thefore managed the secrurity stuff in the all the different IL:s. For the JBoss JMS ra and MDB it might be another matter. We will see. //Peter ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development -- Peter Antman Technology in Media, Box 34105 100 26 Stockholm Systems ArchitectWWW: http://www.tim.se Email: [EMAIL PROTECTED]WWW: http://www.backsource.org Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ and JAAS
On 22 Feb, David Jencks wrote:
On 2002.02.22 14:50:00 -0500 [EMAIL PROTECTED] wrote:
[...]
(Do you remember our discussions earlier about this. I did infact
implement a DirContext impelementation and a DirContext JCA adapter wich
plugged in both as a JAAS LoginModule and made it easy to access user
data through current subject and the DirContext JCA adapter. However I
never committed this, since the DirContext implementation was and it not
based on JBoss naming. I also had to make changes to core JBoss classes
to make this work, and still have problem with the pool implementation
wich does not handle when a resource adapter fails because of
autentication failoure (if configured to wait hard it hangs forever, if
not it leaks connections).
Can you be more specific about what the problem is here?
I'd like to look at your adapter. I've been working (very slowly) on how
to hook up the ConnectionManager to JAAS.
I will try to pack it up and send it to you.
1. I plan to implement the JMS security stuff much the same as the
SecurityInterceptor works. I have made I a first version of an
interceptor layer for JBossMQ where a security interceptor can be
pluged in . No JCA stuff needed here actually.
2. JCA (perhaps) commes in if we raise the question: should current
principal or subject be propagated when opening a JMS connection.
I.e, you obtain a ConnectionFactory in an EJB (through JCA or not),
if you do not use a userid/pwd should the principal/subject be
propagated. Probably yes.
3. Coordinating JCA and LoginModule, so that they use the same
configuration, or perhaps more correct: for a particular JCA we could
JAAS to use it for autentication. As I said above, this does not
work with the current JBoss code base, at least not in 2.4, but it
is possible to fix (and really nice to use).
I think this is what I am working on. I've written a new implementation of
ConnectionManager that implements quite a bit more of the required
functionality and does pooling in (IMHO) a simpler, more pluggable, and
more bug-free way. Hooking up to security is the largest incomplete part.
It's been a while since I worked with it, but I basically did this
(following appendix c - i think - in the JCA spec):
- A JCA RA should provide its own LoginLodule.
- This has to be manually added to auth.conf (or perhaps mor preferably
the xml version of Scott's making.
- The JCA RA specific LoginLodule will save the data it needs to use
later to continue to autenticate the user in specific parts of the
Subject (private credentials).
- I wrote code for my DirCtx to use the same baseclasses to do
autentication both in the LoginLodule and in the ra.
- Next I nedded to get the currentActive subject to be sent to the ra
during connections. I did this by modifying the
ConnectionFactoryLoader and JBossResourceSubjectFactory and having a
SubjectResourceSubjectFactory, which uses the active subject:
public Subject getSubject(ManagedConnectionFactory mcf, String cmName)
{
// For now we go for the Subject directly
// How do we get at the security domain??
try {
return getActiveSubject();
}catch(NamingException ex) {
category.error(Could not get the active subject);
}
return null;
}
private Subject getActiveSubject() throws NamingException{
try {
String domain = null;
InitialContext iniCtx = new InitialContext();
Context securityCtx = (Context) iniCtx.lookup(java:comp/env/security);
JaasSecurityManager securityMgr = (JaasSecurityManager )
securityCtx.lookup(securityMgr);
domain = securityMgr.getSecurityDomain();
Subject s = (Subject) iniCtx.lookup(java:jaas/ + domain+/subject);
category.debug(Returning active subject + s);
return s;
}catch(Exception ex) {
NamingException ne = new NamingException(Could not get current security
domain: + ex.toString());
ne.setRootCause(ex);
throw ne;
}
}
For the ldap/dirCtx stuff, this means that the credentiall will allways
be saved and ne available. At the time I though Scott considered this a
verry bad idea. But now I can see that it is much worse than the
password beeing sent in cleartext in every RMI invokation (if using JAAS
and clear text passwords that is).
I don't think I really solved having a unifyed configuration either,
partly because the auth.conf loading is (if I remember correct) verry
much static and hard to override.
I will send you the code, I we may continue the discussion if you want
to.
//Peter
david jencks
For the JVM invoker (which is only ever used inside the same VM as
JBoss9, do you think it would be a good aproximation to say that a
connection and its child object will allways be accessed by the same
context classloader that
[JBoss-dev] JBossMQ StateManager
Hi, you have plane to implementing JDBC StateManager? //Alexander _ View thread online: http://main.jboss.org/thread.jsp?forum=66thread=9412 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ StateManager
On 22 Feb, Alexander Horuzhiy wrote: Hi, you have plane to implementing JDBC StateManager? No, perhaps make it more pluggable. Then I would probably make an DirContext/Ladap one...but thats for later. First we need to get autentication pluggable. //Peter //Alexander _ View thread online: http://main.jboss.org/thread.jsp?forum=66thread=9412 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development -- Peter Antman Technology in Media, Box 34105 100 26 Stockholm Systems ArchitectWWW: http://www.tim.se Email: [EMAIL PROTECTED]WWW: http://www.backsource.org Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] JBossMQ and JAAS
Hi all, but mostly Scott I guess ;-) I am experimentally adding JAAS support for JBossMQ to handle autentication and autorization. And I have some problem with how to propagate and keep the principall around for the rmi and jvm invokation layers (for OIL and OUL it seems easy beacuse you can use ThreadLocals on the server side). Both the rmi and jvm are however stateless and are represented by only one object each and no thread they personally own. I tried to look at the documentation for how to set up a JAAS aware client, and then looked somewhat into the proxy, jrmp and SecurityInterceptor code, and guess what, it looks to mee as we are sending both the principall and the credentials on every invokation, and also does an authentication (although mostly against the cache) on every invokation. Is this so? If it is so, is this really good? - Sending a possible clear text password on every invokation? - Having to digest the credentialls on every invokation? If I am wrong (I sort of hope I am), could anyone explain to me how the principal is propagated over rmi from the client. Hiram and I have sort of discussed saving a hashed key in the connection token for JMS which could represent a principal, but would that be good practice? For the JVM invoker (which is only ever used inside the same VM as JBoss9, do you think it would be a good aproximation to say that a connection and its child object will allways be accessed by the same context classloader that created it (in that case we could use ContextClassLocal - yes, same as for ThreadLocal, but with classloader as hashkey instead)? //Peter -- Peter Antman Technology in Media, Box 34105 100 26 Stockholm Systems ArchitectWWW: http://www.tim.se Email: [EMAIL PROTECTED]WWW: http://www.backsource.org Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ and JAAS
well I'm not Scott, but I've been working on security for a client. This probably belongs on one of the forums but here are a few answers. Yes the password is sent clear text. Yes you can use encryption or ssl to protect yourself. Yes it is sent every Invocation. For the rmi, look at the MethodInvocation class for jboss 2.4.x or Invocation for jboss 3. On this subject I would like to propose a few things: For JBoss 2.4.x (I haven't tested on JBoss 3 yet), the logout doesn't do anything (really). Scenario: A client connects, logs in, does a remote invocation, is validated, gets a return, logs out, exits. The client again starts up, logs in, does a remote invocation, OK... here it isn't really validated again it is pulled out of the cache, no password validation, no new roles if they exist. Of course this is because of the cache. And I want the cache as long as the client is still logged in under the same session. The consequence of this is that after a person logs in, and until the cache times out, anyone can impersonate that user all he has to know is the userid. (no special password required) I would like to see a token that is sent with the Invocation that is set when the server actually authenticates a user. It is returned to the client and is re-sent every time the client does a remote invocation. UNTIL... the client logs out or exits of course. Next Invocation after a logout or re-login, the Invocation is naked ;) (doesn't have a token), without the token the server knows to invalidate his cache and re-authenticate. Scott, any thoughts? Ken - Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, February 22, 2002 12:26 PM Subject: [JBoss-dev] JBossMQ and JAAS Hi all, but mostly Scott I guess ;-) I am experimentally adding JAAS support for JBossMQ to handle autentication and autorization. And I have some problem with how to propagate and keep the principall around for the rmi and jvm invokation layers (for OIL and OUL it seems easy beacuse you can use ThreadLocals on the server side). Both the rmi and jvm are however stateless and are represented by only one object each and no thread they personally own. I tried to look at the documentation for how to set up a JAAS aware client, and then looked somewhat into the proxy, jrmp and SecurityInterceptor code, and guess what, it looks to mee as we are sending both the principall and the credentials on every invokation, and also does an authentication (although mostly against the cache) on every invokation. Is this so? If it is so, is this really good? - Sending a possible clear text password on every invokation? - Having to digest the credentialls on every invokation? If I am wrong (I sort of hope I am), could anyone explain to me how the principal is propagated over rmi from the client. Hiram and I have sort of discussed saving a hashed key in the connection token for JMS which could represent a principal, but would that be good practice? For the JVM invoker (which is only ever used inside the same VM as JBoss9, do you think it would be a good aproximation to say that a connection and its child object will allways be accessed by the same context classloader that created it (in that case we could use ContextClassLocal - yes, same as for ThreadLocal, but with classloader as hashkey instead)? //Peter -- Peter Antman Technology in Media, Box 34105 100 26 Stockholm Systems ArchitectWWW: http://www.tim.se Email: [EMAIL PROTECTED]WWW: http://www.backsource.org Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ and JAAS
I tried to look at the documentation for how to set up a JAAS aware client, and then looked somewhat into the proxy, jrmp and SecurityInterceptor code, and guess what, it looks to mee as we are sending both the principall and the credentials on every invokation, and also does an authentication (although mostly against the cache) on every invokation. Is this so? Yes. If it is so, is this really good? - Sending a possible clear text password on every invokation? - Having to digest the credentialls on every invokation? This is the stateless web model. You either have to invoke transport level encryption, use finer grained encryption on the sensitive info, or use some opaque transient session key created using a public key exchange algorithm. Transport encryption can be expensive, or it can be free if you have a network that supports TLS at the ethernet card level like Intel does. If I am wrong (I sort of hope I am), could anyone explain to me how the principal is propagated over rmi from the client. By infecting the smart proxy layer obtained from the JNDI home lookup with the credential information established by the JAAS login. Hiram and I have sort of discussed saving a hashed key in the connection token for JMS which could represent a principal, but would that be good practice? David and I have discussed using the JCA facilities for this. Security implementation has to move out of JBossMQ and into the security manager layer. If you want to maintain a usable security model independent of the JBoss server core, then we have to come up with a pluggable model that will work with security through JCA. Let's start discussing this. For the JVM invoker (which is only ever used inside the same VM as JBoss9, do you think it would be a good aproximation to say that a connection and its child object will allways be accessed by the same context classloader that created it (in that case we could use ContextClassLocal - yes, same as for ThreadLocal, but with classloader as hashkey instead)? To do what? ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ and JAAS
This is strictly a login module configuration issue. The SRP login module implementation comes as a client/server pair of login modules that maintain a user based state that can be logged out immeadiately. Read the SRP section in the JBossSX chapter of the book as it talks in detail about this. All is possible with JBoss. Scenario: A client connects, logs in, does a remote invocation, is validated, gets a return, logs out, exits. The client again starts up, logs in, does a remote invocation, OK... here it isn't really validated again it is pulled out of the cache, no password validation, no new roles if they exist. Of course this is because of the cache. And I want the cache as long as the client is still logged in under the same session. The consequence of this is that after a person logs in, and until the cache times out, anyone can impersonate that user all he has to know is the userid. (no special password required) I would like to see a token that is sent with the Invocation that is set when the server actually authenticates a user. It is returned to the client and is re-sent every time the client does a remote invocation. UNTIL... the client logs out or exits of course. Next Invocation after a logout or re-login, the Invocation is naked ;) (doesn't have a token), without the token the server knows to invalidate his cache and re-authenticate. Scott, any thoughts? ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ and JAAS
On 22 Feb, Scott M Stark wrote: I tried to look at the documentation for how to set up a JAAS aware client, and then looked somewhat into the proxy, jrmp and SecurityInterceptor code, and guess what, it looks to mee as we are sending both the principall and the credentials on every invokation, and also does an authentication (although mostly against the cache) on every invokation. Is this so? Yes. If it is so, is this really good? - Sending a possible clear text password on every invokation? - Having to digest the credentialls on every invokation? This is the stateless web model. ? Not if you have a standalone client. The its should be the stateless RMI model, or what? You either have to invoke transport level encryption, use finer grained encryption on the sensitive info, or use some opaque transient session key created using a public key exchange algorithm. Transport encryption can be expensive, or it can be free if you have a network that supports TLS at the ethernet card level like Intel does. The more I think about it, we could start by using a simple session mechanism. I have to think about how to make this not to uncompatible with other auth mechanism then clear text passwords. If I am wrong (I sort of hope I am), could anyone explain to me how the principal is propagated over rmi from the client. By infecting the smart proxy layer obtained from the JNDI home lookup with the credential information established by the JAAS login. Hiram and I have sort of discussed saving a hashed key in the connection token for JMS which could represent a principal, but would that be good practice? David and I have discussed using the JCA facilities for this. Security implementation has to move out of JBossMQ and into the security manager layer. If you want to maintain a usable security model independent of the JBoss server core, then we have to come up with a pluggable model that will work with security through JCA. Let's start discussing this. This includes a lot of stuff. (Do you remember our discussions earlier about this. I did infact implement a DirContext impelementation and a DirContext JCA adapter wich plugged in both as a JAAS LoginModule and made it easy to access user data through current subject and the DirContext JCA adapter. However I never committed this, since the DirContext implementation was and it not based on JBoss naming. I also had to make changes to core JBoss classes to make this work, and still have problem with the pool implementation wich does not handle when a resource adapter fails because of autentication failoure (if configured to wait hard it hangs forever, if not it leaks connections). 1. I plan to implement the JMS security stuff much the same as the SecurityInterceptor works. I have made I a first version of an interceptor layer for JBossMQ where a security interceptor can be pluged in . No JCA stuff needed here actually. 2. JCA (perhaps) commes in if we raise the question: should current principal or subject be propagated when opening a JMS connection. I.e, you obtain a ConnectionFactory in an EJB (through JCA or not), if you do not use a userid/pwd should the principal/subject be propagated. Probably yes. 3. Coordinating JCA and LoginModule, so that they use the same configuration, or perhaps more correct: for a particular JCA we could JAAS to use it for autentication. As I said above, this does not work with the current JBoss code base, at least not in 2.4, but it is possible to fix (and really nice to use). For the JVM invoker (which is only ever used inside the same VM as JBoss9, do you think it would be a good aproximation to say that a connection and its child object will allways be accessed by the same context classloader that created it (in that case we could use ContextClassLocal - yes, same as for ThreadLocal, but with classloader as hashkey instead)? To do what? Well, a connection hold in vm may possibly be used by different threads, but the connection should, in case it was started with userid/pwd, hold ontoit itself, and it ought to be the server side that does this. Instead of relying on a thread local as in SecurityAssociation and the security manager (for subject), we could perhaps store the subject in ClassContextLocal storage. But I am reallt not shure about this now. //Peter ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development -- Peter Antman Technology in Media, Box 34105 100 26 Stockholm Systems ArchitectWWW: http://www.tim.se Email: [EMAIL PROTECTED]WWW: http://www.backsource.org Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 ___
Re: [JBoss-dev] JBossMQ and JAAS
This is the stateless web model. Not if you have a standalone client. The its should be the stateless RMI model, or what? Yes, the RMI proxies are stateless with respect to the client. 2. JCA (perhaps) commes in if we raise the question: should current principal or subject be propagated when opening a JMS connection. I.e, you obtain a ConnectionFactory in an EJB (through JCA or not), if you do not use a userid/pwd should the principal/subject be propagated. Probably yes. This is a definite yes. 3. Coordinating JCA and LoginModule, so that they use the same configuration, or perhaps more correct: for a particular JCA we could JAAS to use it for autentication. As I said above, this does not work with the current JBoss code base, at least not in 2.4, but it is possible to fix (and really nice to use). The new resource adaptor security will only be implemented in 3.x. The current 2.4 code base does not matter. Well, a connection hold in vm may possibly be used by different threads, but the connection should, in case it was started with userid/pwd, hold ontoit itself, and it ought to be the server side that does this. Instead of relying on a thread local as in SecurityAssociation and the security manager (for subject), we could perhaps store the subject in ClassContextLocal storage. But I am reallt not shure about this now. This will be an implementation detail of your resource adaptor. There will be no use of SecurityAssociation at the resource adaptor level. ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ and JAAS
On 2002.02.22 14:50:00 -0500 [EMAIL PROTECTED] wrote: On 22 Feb, Scott M Stark wrote: I tried to look at the documentation for how to set up a JAAS aware client, and then looked somewhat into the proxy, jrmp and SecurityInterceptor code, and guess what, it looks to mee as we are sending both the principall and the credentials on every invokation, and also does an authentication (although mostly against the cache) on every invokation. Is this so? Yes. If it is so, is this really good? - Sending a possible clear text password on every invokation? - Having to digest the credentialls on every invokation? This is the stateless web model. ? Not if you have a standalone client. The its should be the stateless RMI model, or what? You either have to invoke transport level encryption, use finer grained encryption on the sensitive info, or use some opaque transient session key created using a public key exchange algorithm. Transport encryption can be expensive, or it can be free if you have a network that supports TLS at the ethernet card level like Intel does. The more I think about it, we could start by using a simple session mechanism. I have to think about how to make this not to uncompatible with other auth mechanism then clear text passwords. If I am wrong (I sort of hope I am), could anyone explain to me how the principal is propagated over rmi from the client. By infecting the smart proxy layer obtained from the JNDI home lookup with the credential information established by the JAAS login. Hiram and I have sort of discussed saving a hashed key in the connection token for JMS which could represent a principal, but would that be good practice? David and I have discussed using the JCA facilities for this. Security implementation has to move out of JBossMQ and into the security manager layer. If you want to maintain a usable security model independent of the JBoss server core, then we have to come up with a pluggable model that will work with security through JCA. Let's start discussing this. This includes a lot of stuff. (Do you remember our discussions earlier about this. I did infact implement a DirContext impelementation and a DirContext JCA adapter wich plugged in both as a JAAS LoginModule and made it easy to access user data through current subject and the DirContext JCA adapter. However I never committed this, since the DirContext implementation was and it not based on JBoss naming. I also had to make changes to core JBoss classes to make this work, and still have problem with the pool implementation wich does not handle when a resource adapter fails because of autentication failoure (if configured to wait hard it hangs forever, if not it leaks connections). Can you be more specific about what the problem is here? I'd like to look at your adapter. I've been working (very slowly) on how to hook up the ConnectionManager to JAAS. 1. I plan to implement the JMS security stuff much the same as the SecurityInterceptor works. I have made I a first version of an interceptor layer for JBossMQ where a security interceptor can be pluged in . No JCA stuff needed here actually. 2. JCA (perhaps) commes in if we raise the question: should current principal or subject be propagated when opening a JMS connection. I.e, you obtain a ConnectionFactory in an EJB (through JCA or not), if you do not use a userid/pwd should the principal/subject be propagated. Probably yes. 3. Coordinating JCA and LoginModule, so that they use the same configuration, or perhaps more correct: for a particular JCA we could JAAS to use it for autentication. As I said above, this does not work with the current JBoss code base, at least not in 2.4, but it is possible to fix (and really nice to use). I think this is what I am working on. I've written a new implementation of ConnectionManager that implements quite a bit more of the required functionality and does pooling in (IMHO) a simpler, more pluggable, and more bug-free way. Hooking up to security is the largest incomplete part. david jencks For the JVM invoker (which is only ever used inside the same VM as JBoss9, do you think it would be a good aproximation to say that a connection and its child object will allways be accessed by the same context classloader that created it (in that case we could use ContextClassLocal - yes, same as for ThreadLocal, but with classloader as hashkey instead)? To do what? Well, a connection hold in vm may possibly be used by different threads, but the connection should, in case it was started with userid/pwd, hold ontoit itself, and it ought to be the server side that does this. Instead of relying on a thread local as in SecurityAssociation and the security manager (for subject), we could perhaps store the subject in ClassContextLocal storage. But I
Re: [JBoss-dev] JBossMQ test case is soooo slooooow
I will eventually get around to fixing this and in general making the testsuite better. I have been playing with a few ideas to make it easier and faster to write test cases too... if only I had more time. --jason marc fleury wrote: |to have testcases that send up to 100 000 messages, sometimes taking up |to an hour to complete... | |How about that? impractical for the 'fast' test that make sure we don't mess up stuff when we change something. it seems, jason, that we need to use your wonderful buildmagic to get that lenghty test out of there and just run it every night on the beluga machine but take it out of the standard test. marcf | |//Peter | | --jason | | | | | ___ | Jboss-development mailing list | [EMAIL PROTECTED] | https://lists.sourceforge.net/lists/listinfo/jboss-development | |-- | |Peter Antman Technology in Media, Box 34105 100 26 Stockholm |Systems ArchitectWWW: http://www.tim.se |Email: [EMAIL PROTECTED]WWW: http://www.backsource.org |Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 | | | |___ |Jboss-development mailing list |[EMAIL PROTECTED] |https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ test case is soooo slooooow
On 21 Feb, Jason Dillon wrote: I will eventually get around to fixing this and in general making the testsuite better. I have been playing with a few ideas to make it easier and faster to write test cases too... if only I had more time. Oh, yes, time. I'am sitting here with this faboulous (;-)) idea of how to rewrite JBossMQ to also use the MBuss/JBoss decorator pattern...oh, yes, time... I have made a new set of stress and massive tests for JBossMQ that I hope to commit sometime next week, and I might be able to take a quick look att the complete suite of JMS tests (mdb,jmsra and jbossmq), since I have a couple of nice JMS worker basec lasses that makes it a litle bit easier to write stable JMS testcases. By the way, these new tests are verry time consuming, they starts and stops JBoss and use big iteration counts. My plan is to integrate is as an entity ref in build.xml, to not litter that file to much, and not make them run dring normal testing. Is that OK? By the way, I need to get the information in the build where the JBoss output build is. Is there any way to extract this information from build/build.xml when running testsuite/build.xml (I mean, build magic way of doing it)? //Peter --jason marc fleury wrote: |to have testcases that send up to 100 000 messages, sometimes taking up |to an hour to complete... | |How about that? impractical for the 'fast' test that make sure we don't mess up stuff when we change something. it seems, jason, that we need to use your wonderful buildmagic to get that lenghty test out of there and just run it every night on the beluga machine but take it out of the standard test. marcf | |//Peter | | --jason | | | | | ___ | Jboss-development mailing list | [EMAIL PROTECTED] | https://lists.sourceforge.net/lists/listinfo/jboss-development | |-- | |Peter Antman Technology in Media, Box 34105 100 26 Stockholm |Systems ArchitectWWW: http://www.tim.se |Email: [EMAIL PROTECTED]WWW: http://www.backsource.org |Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 | | | |___ |Jboss-development mailing list |[EMAIL PROTECTED] |https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development -- Peter AntmanChief Systems Architect, Business Development Technology in Media, Box 34105 100 26 Stockholm WWW: http://www.tim.se WWW: http://www.backsource.org Email: [EMAIL PROTECTED] Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ test case is soooo slooooow
By the way, these new tests are verry time consuming, they starts and stops JBoss and use big iteration counts. My plan is to integrate is as an entity ref in build.xml, to not litter that file to much, and not make them run dring normal testing. Is that OK? I am not sure what your changes include... so commit as enity-ref and if it is a problem or can be done in a better way we can deal with it then. By the way, I need to get the information in the build where the JBoss output build is. Is there any way to extract this information from build/build.xml when running testsuite/build.xml (I mean, build magic way of doing it)? Currently no... because there is no telling if the local build.xml was used or if the build/build.xml was used. I could probably write a bit of BM to fix this though (a task which would check if there is a ../build/build.xml and parse it and import some properties). Again we come back to time... --jason ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] JBossMQ test case is soooo slooooow
|to have testcases that send up to 100 000 messages, sometimes taking up |to an hour to complete... | |How about that? impractical for the 'fast' test that make sure we don't mess up stuff when we change something. it seems, jason, that we need to use your wonderful buildmagic to get that lenghty test out of there and just run it every night on the beluga machine but take it out of the standard test. marcf | |//Peter | | --jason | | | | | ___ | Jboss-development mailing list | [EMAIL PROTECTED] | https://lists.sourceforge.net/lists/listinfo/jboss-development | |-- | |Peter Antman Technology in Media, Box 34105 100 26 Stockholm |Systems ArchitectWWW: http://www.tim.se |Email: [EMAIL PROTECTED]WWW: http://www.backsource.org |Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 | | | |___ |Jboss-development mailing list |[EMAIL PROTECTED] |https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ test case is soooo slooooow
Hi, one reason it is taking so long is that it is testing all IL:s (I thingk). If you do this move please split this test and run at least a test on the OIL and preferably also the JVM IL, since these are the defaults used by everyone. //Peter On 20 Feb, marc fleury wrote: |to have testcases that send up to 100 000 messages, sometimes taking up |to an hour to complete... | |How about that? impractical for the 'fast' test that make sure we don't mess up stuff when we change something. it seems, jason, that we need to use your wonderful buildmagic to get that lenghty test out of there and just run it every night on the beluga machine but take it out of the standard test. marcf | |//Peter | | --jason | | | | | ___ | Jboss-development mailing list | [EMAIL PROTECTED] | https://lists.sourceforge.net/lists/listinfo/jboss-development | |-- | |Peter Antman Technology in Media, Box 34105 100 26 Stockholm |Systems ArchitectWWW: http://www.tim.se |Email: [EMAIL PROTECTED]WWW: http://www.backsource.org |Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 | | | |___ |Jboss-development mailing list |[EMAIL PROTECTED] |https://lists.sourceforge.net/lists/listinfo/jboss-development -- Peter Antman Technology in Media, Box 34105 100 26 Stockholm Systems ArchitectWWW: http://www.tim.se Email: [EMAIL PROTECTED]WWW: http://www.backsource.org Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ test case is soooo slooooow
On 16 Feb, Jason Dillon wrote: Is there anything we can do to speed it up any? Well, actually it is not slow enough ;-) since it does not catch all bugs as it is setup today. To chatch current buggs in threading you have to have testcases that send up to 100 000 messages, sometimes taking up to an hour to complete... How about that? //Peter --jason ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development -- Peter Antman Technology in Media, Box 34105 100 26 Stockholm Systems ArchitectWWW: http://www.tim.se Email: [EMAIL PROTECTED]WWW: http://www.backsource.org Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ test case is soooo slooooow
Yikes... that would not bee good. Perhaps this test case should be moved outside of tests-unit into a separate target which is depended on by tests. --jason On Sun, 2002-02-17 at 07:22, [EMAIL PROTECTED] wrote: On 16 Feb, Jason Dillon wrote: Is there anything we can do to speed it up any? Well, actually it is not slow enough ;-) since it does not catch all bugs as it is setup today. To chatch current buggs in threading you have to have testcases that send up to 100 000 messages, sometimes taking up to an hour to complete... How about that? //Peter --jason ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development -- Peter Antman Technology in Media, Box 34105 100 26 Stockholm Systems ArchitectWWW: http://www.tim.se Email: [EMAIL PROTECTED]WWW: http://www.backsource.org Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] JBossMQ test case is soooo slooooow
Is there anything we can do to speed it up any? --jason ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq message transport times
* The test I'm running is using 'localhost' (at least, I think so-- it uses the same infrastructure as the existing perf tests in the testsuite module). * It perhaps isn't surprising that MacOS X has TCPNODELAY defaulted to false (and I bet MacOS Server doesn't). What had me puzzled was why that would affect loopback tests. I might mention that earlier testing with varying packet sizes did indeed show a stairstep response time with 200 ms increases at each jump (in fact it wasn't just a random dart throw that motived my change to the TCPNODELAY). * Even creating an empty message creates a packet with more than 200 bytes-- due to the overhead for the message id and so forth. On the other hand, there is the application level acknowledgement packet, which is indeed a tinygram. * There is likely to be a bunch of work needed if we really want to support high performance message traffic over high latency and/or low bandwidth links. Changing the buffer sizes isn't the half of it. Jeff Tulley wrote: Ok, I just got the run-down on buffer sizes and TCPNODELAY. The answer is, of course, IT DEPENDS. If what you are sending out always will be 100 or 200 bytes, the buffer size will not affect you very much - you were sending tiny-grams out before and will continue to do so even after setting TCPNODELAY. The only difference is that before you were only sending a tiny-gram out every 200-400ms (good from a network traffic point of view, very BAD from a developer's point of view). Then again, if these small messages are few and far between, I wouldn't sweat sending out the small packets at all. If you have a feel for typical MQ packet sizes you can come up with a better buffer size. For instance, even if most messages are 2-4K, if you have occasional ones at 10K, then set your buffer to 10K. The extra buffer is most likely worth the increase in efficiency. (Here again there is a fine line between efficiency and wasted, unused buffer space). Actually, further refinement: The buffer size for TCP is 1460 bytes. So, try to set your buffer size to a multiple of this, so instead of 16K you would do 16 x 1460 = 23360 bytes. That way you are maximizing the chance that all data out will completely fill up multiple packets instead of having a few full and then a partially empty packet every time. I would not go much above this 16 multiplier unless all data sent is always way larger than that, but this is no hard rule of thumb. Complicated enough? It definitely will be worth it to implement this. 1000's of packets per second vs 5 packets per second(or as Loren saw, about 2.5 per second) is a HUGE deal. Jeff Tulley ([EMAIL PROTECTED]) (801)861-5322 Novell, Inc., the leading provider of Net business solutions http://www.novell.com Hiram Chirino [EMAIL PROTECTED] 1/16/02 1:59:34 PM So, Are you saying that if we do TCPNODELAY, just make sure we setup a BIG BufferedOutputStream and make sure we got our flush()es in the right places??? The question is: how BIG should the buffer be??? Regards, Hiram From: Jeff Tulley [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: RE: [JBoss-dev] jbossmq message transport times Date: Wed, 16 Jan 2002 10:03:23 -0700 Actually it is even a little more complicated than that. Nagle's algorithm by itself would not cause such delays, but what happened is that at the other end somebody (I forget who) came up with the idea of a delayed ACK. Since an ACK is a small packet, and one ACK can acknowledge receipt for multiple packets, the delayed ACK algorithm tries to send an ACK only every other packet. This saves the wire from having too many tini-gram ACK packets. Combined with the Nagle algorithm, this causes death of communication since neither side will budge Eventually the delayed ACK algorithm times out (typically 200 ms, depending on the OS), and the ACK is sent. So, you get down to about 5 packets per second throughput. There are two other algorithms, which modify NaglReceived: from INET-PRV-MTA by prv-mail20.provo.novell.com wite's to get around this problem. One, the Doupnik algorithm, uses information from the data to determine if there is more coming, and it sends the last bit immediately without waiting for the buffer to fill up. So, you get full packets until the last one, which will be a tiny gram, but you do not get the 200 ms delay for every packet since the receiving end is continually getting information. See http://netlab1.usu.edu/pub/misc/draft-doupnik-tcpimpl-nagle-mode-00.txt for information. The other algorithm is called the Minshall Algorithm. I do not know a lot about it, but it is along the same lines. See http://lists.w3.org/Archives/Public/ietf-discuss/1998Dec/0025.html for a discussion. The problem with setting TCPNODELAY is that it disables Nagle's altogether and you get the wire flooded with a lot of tiny packets. But, in the web space it is exactly
Re: [JBoss-dev] jbossmq message transport times
I made two changes which I'll describe here. I haven't yet investigated the
mechanics of submitting patches.
The changes are both in the org.jboss.mq.il.oil package. Note there are analogous
places in the other il subpackages that may also need changing-- my current test
evidently doesn't exercise the other code paths.
In OILServerIL, add a line to the createConnection method:
private void createConnection()
throws Exception
{
socket = new Socket(addr, port);
// add the next line
socket.setTcpNoDelay(true);
in = new ObjectInputStream(new
BufferedInputStream(socket.getInputStream()));
out = new ObjectOutputStream(new
BufferedOutputStream(socket.getOutputStream()));
out.flush();
}
The same line gets added in OILServerILService
public void run()
{
try
{
while (running)
{
Socket socket = null;
try
{
socket = serverSocket.accept();
}
catch (java.io.InterruptedIOException e)
{
continue;
}
// it's possible that the service is no longer
// running but it got a connection, no point in
// starting up a thread!
//
if (!running)
{
if(socket != null)
{
try
{
socket.close();
}
catch(Exception e)
{
// do nothing
}
}
return;
}
try
{
socket.setSoTimeout(0);
// add next line
socket.setTcpNoDelay(true);
new Thread(new Client(socket), OIL Worker).start();
}
catch(IOException ie)
{
if(log.isDebugEnabled())
{
log.debug(IOException processing client connection, ie);
log.debug(Dropping client connection, server will not
terminate);
}
}
}
}
catch (SocketException e)
{
// There is no easy way (other than string comparison) to
// determine if the socket exception is caused by connection
// reset by peer. In this case, it's okay to ignore both
// SocketException and IOException.
log.warn(SocketException occured (Connection reset by peer?). Cannot
initialize the OILServerILService.);
}
catch (IOException e)
{
log.warn(IOException occured. Cannot initialize the
OILServerILService.);
}
finally
{
try
{
serverSocket.close();
}
catch(Exception e)
{
log.debug(error closing server socket, e);
}
return;
}
}
Christian Riege wrote:
hi,
On Wed, 2002-01-16 at 10:58, Sacha Labourey wrote:
I guess that if you want to modify this, you need to make it optional.
The TCPNODELAY flag is related to the Nagle's algorithm
true.
This algorithm is made to avoid sending very small paquets each time you
send data through your socket [...]
primarily useful for telnet/ssh style applications.
I am not familiar with JBossMQ code base but if you think that the
implementation could, at some time, send many small messages and would
JBossMQ sends packets whenever a message needs to be delivered via the
IL (except for the InVM layer but we can put that aside).
suffer from sending each time a paquet, then you should most probably make
this setting optional. But, on the opposite, if you think that your paquet
(*all* paquets, even ACK) always have a sufficient size and don't want Nagle
to take care of you, then it is up to you.
... don't forget the new pinger thread which tries to ping the server
every once in a while to ensure its still up and running.
bottom line: TCP_NODELAY should be set to *enabled* whenever you're in a
low latency / high bandwidth situation, i.e. LAN or something similiar.
I guess this applies to at the very least 80% of the JBossMQ
installations out there.
I would disable TCP_NODELAY only when being in a very low bandwidth /
high latency situation.
Loren, can you pls. submit a patch via SourceForge (or send it to this
list) and we'll get this integrated into the codebase.
Regarding Mac OS/X in this situation: it could very well be that their
default mode of operation is to have TCP_NODELAY disabled by default,
even when running on the loopback device -- you ARE using 127.0.0.1
after all, right?. On my Linux box this is a switch when configuring the
kernel (at least it has been when I was still building my own kernels
... which hasn't happened since 2.0.something).
Regards,
Christian
___
Jboss-development mailing list
[EMAIL
Re: [JBoss-dev] jbossmq message transport times
The TcpNoDelay flag should be a configurable attribute
exposed through the org.jboss.mq.il.oil.OILServerILServiceMBean.
Patches should be submitted through sourceforge as cvs diffs.
Scott Stark
Chief Technology Officer
JBoss Group, LLC
- Original Message -
From: Loren Rosen [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Friday, January 18, 2002 1:58 PM
Subject: Re: [JBoss-dev] jbossmq message transport times
I made two changes which I'll describe here. I haven't yet investigated
the
mechanics of submitting patches.
The changes are both in the org.jboss.mq.il.oil package. Note there are
analogous
places in the other il subpackages that may also need changing-- my
current test
evidently doesn't exercise the other code paths.
In OILServerIL, add a line to the createConnection method:
private void createConnection()
throws Exception
{
socket = new Socket(addr, port);
// add the next line
socket.setTcpNoDelay(true);
___
Jboss-development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] jbossmq message transport times
Hello, I guess that if you want to modify this, you need to make it optional. The TCPNODELAY flag is related to the Nagle's algorithm This algorithm is made to avoid sending very small paquets each time you send data through your socket but instead wait 400ms (generally) or a full buffer (or a message from the other side if I remember well) and send a paquet that wrap more data. Thus, it tends to minimize the amount of small data that are sent. I am not familiar with JBossMQ code base but if you think that the implementation could, at some time, send many small messages and would suffer from sending each time a paquet, then you should most probably make this setting optional. But, on the opposite, if you think that your paquet (*all* paquets, even ACK) always have a sufficient size and don't want Nagle to take care of you, then it is up to you. Cheers, Sacha -Message d'origine- De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]De la part de Hiram Chirino Envoyé : mercredi, 16 janvier 2002 04:17 À : [EMAIL PROTECTED]; [EMAIL PROTECTED] Objet : Re: [JBoss-dev] jbossmq message transport times I can't think of a reason that it would break anything... ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq message transport times
Den 2002-01-16 02:37:12 skrev Loren Rosen [EMAIL PROTECTED]: I'm testing on MacOS X, which for our purposes is just another Unix flavor with an oddball GUI. Hi hi hi I'm waiting for my PB TI to arrive... really longing for Unix with an oddball GUI :) /Lennart ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] jbossmq message transport times
hi, On Wed, 2002-01-16 at 10:58, Sacha Labourey wrote: I guess that if you want to modify this, you need to make it optional. The TCPNODELAY flag is related to the Nagle's algorithm true. This algorithm is made to avoid sending very small paquets each time you send data through your socket [...] primarily useful for telnet/ssh style applications. I am not familiar with JBossMQ code base but if you think that the implementation could, at some time, send many small messages and would JBossMQ sends packets whenever a message needs to be delivered via the IL (except for the InVM layer but we can put that aside). suffer from sending each time a paquet, then you should most probably make this setting optional. But, on the opposite, if you think that your paquet (*all* paquets, even ACK) always have a sufficient size and don't want Nagle to take care of you, then it is up to you. ... don't forget the new pinger thread which tries to ping the server every once in a while to ensure its still up and running. bottom line: TCP_NODELAY should be set to *enabled* whenever you're in a low latency / high bandwidth situation, i.e. LAN or something similiar. I guess this applies to at the very least 80% of the JBossMQ installations out there. I would disable TCP_NODELAY only when being in a very low bandwidth / high latency situation. Loren, can you pls. submit a patch via SourceForge (or send it to this list) and we'll get this integrated into the codebase. Regarding Mac OS/X in this situation: it could very well be that their default mode of operation is to have TCP_NODELAY disabled by default, even when running on the loopback device -- you ARE using 127.0.0.1 after all, right?. On my Linux box this is a switch when configuring the kernel (at least it has been when I was still building my own kernels ... which hasn't happened since 2.0.something). Regards, Christian ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] jbossmq message transport times
Actually it is even a little more complicated than that. Nagle's algorithm by itself would not cause such delays, but what happened is that at the other end somebody (I forget who) came up with the idea of a delayed ACK. Since an ACK is a small packet, and one ACK can acknowledge receipt for multiple packets, the delayed ACK algorithm tries to send an ACK only every other packet. This saves the wire from having too many tini-gram ACK packets. Combined with the Nagle algorithm, this causes death of communication since neither side will budge Eventually the delayed ACK algorithm times out (typically 200 ms, depending on the OS), and the ACK is sent. So, you get down to about 5 packets per second throughput. There are two other algorithms, which modify NaglReceived: from INET-PRV-MTA by prv-mail20.provo.novell.com wite's to get around this problem. One, the Doupnik algorithm, uses information from the data to determine if there is more coming, and it sends the last bit immediately without waiting for the buffer to fill up. So, you get full packets until the last one, which will be a tiny gram, but you do not get the 200 ms delay for every packet since the receiving end is continually getting information. See http://netlab1.usu.edu/pub/misc/draft-doupnik-tcpimpl-nagle-mode-00.txt for information. The other algorithm is called the Minshall Algorithm. I do not know a lot about it, but it is along the same lines. See http://lists.w3.org/Archives/Public/ietf-discuss/1998Dec/0025.html for a discussion. The problem with setting TCPNODELAY is that it disables Nagle's altogether and you get the wire flooded with a lot of tiny packets. But, in the web space it is exactly these tiny packets which trigger the problem, and otherwise you lose throughput. The best solution, lacking the ability to change your OS TCP stack, is to turn Nagle's off (with the TCPNODELAY), and then use a bigger buffer for your data. Then you are using the wire more efficiently in that you are not flooding it with tiny-grams. I wouldn't know all of this except the coworker in the office next to me took a class from Doupnik himself, and has recently just finished working with NetWare's TCP engineers to implement the Doupnik algorithm. He found that most programs he dealt with in testing for the problem turned off Nagle's algorithm for efficiency. Some OS's have this fix in them, some do not. (Most do not???) If we turn off Nagles algorithm in that code, we defintely need to look at the buffer size if we want the most efficient data transfer. Jeff Tulley ([EMAIL PROTECTED]) (801)861-5322 Novell, Inc., the leading provider of Net business solutions http://www.novell.com Sacha Labourey [EMAIL PROTECTED] 1/16/02 2:58:30 AM Hello, I guess that if you want to modify this, you need to make it optional. The TCPNODELAY flag is related to the Nagle's algorithm This algorithm is made to avoid sending very small paquets each time you send data through your socket but instead wait 400ms (generally) or a full buffer (or a message from the other side if I remember well) and send a paquet that wrap more data. Thus, it tends to minimize the amount of small data that are sent. I am not familiar with JBossMQ code base but if you think that the implementation could, at some time, send many small messages and would suffer from sending each time a paquet, then you should most probably make this setting optional. But, on the opposite, if you think that your paquet (*all* paquets, even ACK) always have a sufficient size and don't want Nagle to take care of you, then it is up to you. Cheers, Sacha -Message d'origine- De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]De la part de Hiram Chirino Envoyé : mercredi, 16 janvier 2002 04:17 À : [EMAIL PROTECTED]; [EMAIL PROTECTED] Objet : Re: [JBoss-dev] jbossmq message transpo rt times I can't think of a reason that it would break anything... ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] jbossmq message transport times
So, Are you saying that if we do TCPNODELAY, just make sure we setup a BIG BufferedOutputStream and make sure we got our flush()es in the right places??? The question is: how BIG should the buffer be??? Regards, Hiram From: Jeff Tulley [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: RE: [JBoss-dev] jbossmq message transport times Date: Wed, 16 Jan 2002 10:03:23 -0700 Actually it is even a little more complicated than that. Nagle's algorithm by itself would not cause such delays, but what happened is that at the other end somebody (I forget who) came up with the idea of a delayed ACK. Since an ACK is a small packet, and one ACK can acknowledge receipt for multiple packets, the delayed ACK algorithm tries to send an ACK only every other packet. This saves the wire from having too many tini-gram ACK packets. Combined with the Nagle algorithm, this causes death of communication since neither side will budge Eventually the delayed ACK algorithm times out (typically 200 ms, depending on the OS), and the ACK is sent. So, you get down to about 5 packets per second throughput. There are two other algorithms, which modify NaglReceived: from INET-PRV-MTA by prv-mail20.provo.novell.com wite's to get around this problem. One, the Doupnik algorithm, uses information from the data to determine if there is more coming, and it sends the last bit immediately without waiting for the buffer to fill up. So, you get full packets until the last one, which will be a tiny gram, but you do not get the 200 ms delay for every packet since the receiving end is continually getting information. See http://netlab1.usu.edu/pub/misc/draft-doupnik-tcpimpl-nagle-mode-00.txt for information. The other algorithm is called the Minshall Algorithm. I do not know a lot about it, but it is along the same lines. See http://lists.w3.org/Archives/Public/ietf-discuss/1998Dec/0025.html for a discussion. The problem with setting TCPNODELAY is that it disables Nagle's altogether and you get the wire flooded with a lot of tiny packets. But, in the web space it is exactly these tiny packets which trigger the problem, and otherwise you lose throughput. The best solution, lacking the ability to change your OS TCP stack, is to turn Nagle's off (with the TCPNODELAY), and then use a bigger buffer for your data. Then you are using the wire more efficiently in that you are not flooding it with tiny-grams. I wouldn't know all of this except the coworker in the office next to me took a class from Doupnik himself, and has recently just finished working with NetWare's TCP engineers to implement the Doupnik algorithm. He found that most programs he dealt with in testing for the problem turned off Nagle's algorithm for efficiency. Some OS's have this fix in them, some do not. (Most do not???) If we turn off Nagles algorithm in that code, we defintely need to look at the buffer size if we want the most efficient data transfer. Jeff Tulley ([EMAIL PROTECTED]) (801)861-5322 Novell, Inc., the leading provider of Net business solutions http://www.novell.com Sacha Labourey [EMAIL PROTECTED] 1/16/02 2:58:30 AM Hello, I guess that if you want to modify this, you need to make it optional. The TCPNODELAY flag is related to the Nagle's algorithm This algorithm is made to avoid sending very small paquets each time you send data through your socket but instead wait 400ms (generally) or a full buffer (or a message from the other side if I remember well) and send a paquet that wrap more data. Thus, it tends to minimize the amount of small data that are sent. I am not familiar with JBossMQ code base but if you think that the implementation could, at some time, send many small messages and would suffer from sending each time a paquet, then you should most probably make this setting optional. But, on the opposite, if you think that your paquet (*all* paquets, even ACK) always have a sufficient size and don't want Nagle to take care of you, then it is up to you. Cheers, Sacha -Message d'origine- De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]De la part de Hiram Chirino Envoyé : mercredi, 16 janvier 2002 04:17 À : [EMAIL PROTECTED]; [EMAIL PROTECTED] Objet : Re: [JBoss-dev] jbossmq message transpo rt times I can't think of a reason that it would break anything... ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp. ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo
RE: [JBoss-dev] jbossmq message transport times
Ok, I just got the run-down on buffer sizes and TCPNODELAY. The answer is, of course, IT DEPENDS. If what you are sending out always will be 100 or 200 bytes, the buffer size will not affect you very much - you were sending tiny-grams out before and will continue to do so even after setting TCPNODELAY. The only difference is that before you were only sending a tiny-gram out every 200-400ms (good from a network traffic point of view, very BAD from a developer's point of view). Then again, if these small messages are few and far between, I wouldn't sweat sending out the small packets at all. If you have a feel for typical MQ packet sizes you can come up with a better buffer size. For instance, even if most messages are 2-4K, if you have occasional ones at 10K, then set your buffer to 10K. The extra buffer is most likely worth the increase in efficiency. (Here again there is a fine line between efficiency and wasted, unused buffer space). Actually, further refinement: The buffer size for TCP is 1460 bytes. So, try to set your buffer size to a multiple of this, so instead of 16K you would do 16 x 1460 = 23360 bytes. That way you are maximizing the chance that all data out will completely fill up multiple packets instead of having a few full and then a partially empty packet every time. I would not go much above this 16 multiplier unless all data sent is always way larger than that, but this is no hard rule of thumb. Complicated enough? It definitely will be worth it to implement this. 1000's of packets per second vs 5 packets per second(or as Loren saw, about 2.5 per second) is a HUGE deal. Jeff Tulley ([EMAIL PROTECTED]) (801)861-5322 Novell, Inc., the leading provider of Net business solutions http://www.novell.com Hiram Chirino [EMAIL PROTECTED] 1/16/02 1:59:34 PM So, Are you saying that if we do TCPNODELAY, just make sure we setup a BIG BufferedOutputStream and make sure we got our flush()es in the right places??? The question is: how BIG should the buffer be??? Regards, Hiram From: Jeff Tulley [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: RE: [JBoss-dev] jbossmq message transport times Date: Wed, 16 Jan 2002 10:03:23 -0700 Actually it is even a little more complicated than that. Nagle's algorithm by itself would not cause such delays, but what happened is that at the other end somebody (I forget who) came up with the idea of a delayed ACK. Since an ACK is a small packet, and one ACK can acknowledge receipt for multiple packets, the delayed ACK algorithm tries to send an ACK only every other packet. This saves the wire from having too many tini-gram ACK packets. Combined with the Nagle algorithm, this causes death of communication since neither side will budge Eventually the delayed ACK algorithm times out (typically 200 ms, depending on the OS), and the ACK is sent. So, you get down to about 5 packets per second throughput. There are two other algorithms, which modify NaglReceived: from INET-PRV-MTA by prv-mail20.provo.novell.com wite's to get around this problem. One, the Doupnik algorithm, uses information from the data to determine if there is more coming, and it sends the last bit immediately without waiting for the buffer to fill up. So, you get full packets until the last one, which will be a tiny gram, but you do not get the 200 ms delay for every packet since the receiving end is continually getting information. See http://netlab1.usu.edu/pub/misc/draft-doupnik-tcpimpl-nagle-mode-00.txt for information. The other algorithm is called the Minshall Algorithm. I do not know a lot about it, but it is along the same lines. See http://lists.w3.org/Archives/Public/ietf-discuss/1998Dec/0025.html for a discussion. The problem with setting TCPNODELAY is that it disables Nagle's altogether and you get the wire flooded with a lot of tiny packets. But, in the web space it is exactly these tiny packets which trigger the problem, and otherwise you lose throughput. The best solution, lacking the ability to change your OS TCP stack, is to turn Nagle's off (with the TCPNODELAY), and then use a bigger buffer for your data. Then you are using the wire more efficiently in that you are not flooding it with tiny-grams. I wouldn't know all of this except the coworker in the office next to me took a class from Doupnik himself, and has recently just finished working with NetWare's TCP engineers to implement the Doupnik algorithm. He found that most programs he dealt with in testing for the problem turned off Nagle's algorithm for efficiency. Some OS's have this fix in them, some do not. (Most do not???) If we turn off Nagles algorithm in that code, we defintely need to look at the buffer size if we want the most efficient data transfer. Jeff Tulley ([EMAIL PROTECTED]) (801)861-5322 Novell, Inc., the leading provider of Net business solutions http://www.novell.com Sacha Labourey [EMAIL PROTECTED] 1/16/02 2:58:30 AM Hello, I guess that if you want
[JBoss-dev] jbossmq message transport times
As a start on measuring jbossmq performance, I timed sending and receiving a 1-kbyte nondurable mesage (in a loop to amortize JIT effects and the like). Both client and server were on the same machine. The average round trip time was 400 miliseconds, which is not good at all. Some investigation showed that most of the time was spent idling in the message transport code. I tried a few things and eventually discovered that explicitly setting TCPNODELAY on the sockets fixed the problem-- round trip times dropped down to a few tens of milliseconds. I can certainly submit a patch with the one-line change, but before I do-- is this really the right fix? Will it make something else worse? If no one else has seen the same problem, perhaps it's O/S specific. I'm testing on MacOS X, which for our purposes is just another Unix flavor with an oddball GUI. _ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] jbossmq startup... do we really need ALL these queues by default?
yo, Half the log is with the mq stuff doing queues from A-Z do we *really* need all of these, just cosmetics but still... __ View: http://jboss.org/forums/thread.jsp?forum=66thread=5182 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq startup... do we really need ALL these queuesby default?
Probably not for the default config... some are used for the testsuite, but I think we should probably setup a 'testsuite' config, which is used to run the testsuite against. Downside to this is we may miss configuration errors in the 'default' config. Does the remote queue/topic install work? Could use that to setup the queues/topics required for the tests. --jason Half the log is with the mq stuff doing queues from A-Z do we *really* need all of these, just cosmetics but still... ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq: persistence implementation questions
I'll look through the code more closely. But recovery is testable. What you have to do is create some corrupt data files (probably by using a byte-level editor to munge some existing data files). Then you need some scripts to start up the server with those files -- which doesn't fit so well with a pure-Java Junit-based test suite. (You could also create the files by making versions of the server which quit at various awkward places.) What this won't test so easily is recovery from failures during recovery. How about using JUnit to corrupt some files, then startup the component which reads those files and test that is throws the proper exceptions? --jason ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq: persistence implementation questions
Corrupting the files is not exactly what we are looking for. If you have currupted data files, you need will need to restore you data from backup. What we want to test is: (1) that we are correctly flushing out our output to disk when we commit a transaction. All message are FULLY written to disk by the time the commit is issued. (2) that we do not attempt to recover messages from transaction that did not commit. (3) that we can deal with garbage at the end of out transaction logs. So would adding garbage at the end of a message log files or adding garbage message files simulate(2,3)?? Would HARD Killing the server after a commit test (2)??? How would we implement the HARD kill (JNI)?? Any other tests/ideas?? Regards, Hiram From: Jason Dillon [EMAIL PROTECTED] To: Loren Rosen [EMAIL PROTECTED] CC: Hiram Chirino [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: Re: [JBoss-dev] jbossmq: persistence implementation questions Date: Thu, 15 Nov 2001 15:07:23 -0800 (PST) I'll look through the code more closely. But recovery is testable. What you have to do is create some corrupt data files (probably by using a byte-level editor to munge some existing data files). Then you need some scripts to start up the server with those files -- which doesn't fit so well with a pure-Java Junit-based test suite. (You could also create the files by making versions of the server which quit at various awkward places.) What this won't test so easily is recovery from failures during recovery. How about using JUnit to corrupt some files, then startup the component which reads those files and test that is throws the proper exceptions? --jason _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq: persistence implementation questions
In jboss 3, you __should__ be able to start and stop jbossmq dynamically while the rest of the server is running. We are not very far from being able to run several JBossMQService mbeans at once-- this could allow you to test one pm at a time without messing up the standard jbossmq setup. I think stopping works less well than starting at the moment. You should be able to do this by deploying/undeploying a file like jbossmq-service.xml. There are lots of examples in the testsuite using the JBossTestCase and JBossTestSetup classes: especially the jmx tests. (these don't all pass at the moment, but do show how to deploy/undeploy service files). david jencks On 2001.11.15 17:27:44 -0500 Loren Rosen wrote: Hiram Chirino wrote: Hi Loren! From: Loren Rosen [EMAIL PROTECTED] To: [EMAIL PROTECTED] [EMAIL PROTECTED] Subject: [JBoss-dev] jbossmq: persistance implementation questions Date: Mon, 12 Nov 2001 20:22:27 -0800 * Performance tuning/scalability: has any work been done to address potential bottlenecks for high message throughput, large numbers of queues or topics, etc.? Work has gone into making reading/writing from storage more efficient. I don't think we have profiled the system enough yet to find out where the bottle necks are. A good first step would be to measure the single producer/consumer latency, and tune that. Low latency is a important goal in its own right, and moreover if the CPU overhead is too high you won't even be able to keep the disk busy writing messages. I can go ahead and do this. * recovery robustness: recovery is a tricky thing since there's so many possible ways things can fail. What's been done to think about/test-for/defend-against all the possibilities? Basic recovery test have been succeeding. It's hard to automate failure testing. More code review would be good in this area to see if we are missing anything. I'll look through the code more closely. But recovery is testable. What you have to do is create some corrupt data files (probably by using a byte-level editor to munge some existing data files). Then you need some scripts to start up the server with those files -- which doesn't fit so well with a pure-Java Junit-based test suite. (You could also create the files by making versions of the server which quit at various awkward places.) What this won't test so easily is recovery from failures during recovery. Regards, Hiram _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development _ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq: persistence implementation questions
Use mock objects. www.mockobjects.com To use the automated mock object creator may require modifying the code a little (creating some more interfaces), but you can simply use the concept. It is the only great way to test complicated failure cases, especially when there is a complicated state that has to be established. The idea is to have an object that you can tell it what state to be in - for instance, to fail whenever you call a certain routine, etc, and what kind of failure. It implements the same interface as the real object, so that you can use it interchangeably. There are some other concepts with mockobjects also, such as input or call validation - the object can keep track of how it was called and with what parameters and in what order. This can all be validated against your expectations. the mock objects people call it endotesting because you are testing from the inside, but without tainting your actual code with any knowledge of the testing framework. (Besides the need to create an interface based on the object, which is a sometimes a good thing anyway, though it does add one more file to maintain). Pretty good stuff... Jeff Tulley ([EMAIL PROTECTED]) (801)861-5322 Novell, Inc., the leading provider of Net services software. Loren Rosen [EMAIL PROTECTED] 11/15/01 3:27:44 PM Hiram Chirino wrote: Hi Loren! From: Loren Rosen [EMAIL PROTECTED] To: [EMAIL PROTECTED] [EMAIL PROTECTED] Subject: [JBoss-dev] jbossmq: persistance implementation questions Date: Mon, 12 Nov 2001 20:22:27 -0800 * Performance tuning/scalability: has any work been done to address potential bottlenecks for high message throughput, large numbers of queues or topics, etc.? Work has gone into making reading/writing from storage more efficient. I don't think we have profiled the system enough yet to find out where the bottle necks are. A good first step would be to measure the single producer/consumer latency, and tune that. Low latency is a important goal in its own right, and moreover if the CPU overhead is too high you won't even be able to keep the disk busy writing messages. I can go ahead and do this. * recovery robustness: recovery is a tricky thing since there's so many possible ways things can fail. What's been done to think about/test-for/defend-against all the possibilities? Basic recovery test have been succeeding. It's hard to automate failure testing. More code review would be good in this area to see if we are missing anything. I'll look through the code more closely. But recovery is testable. What you have to do is create some corrupt data files (probably by using a byte-level editor to munge some existing data files). Then you need some scripts to start up the server with those files -- which doesn't fit so well with a pure-Java Junit-based test suite. (You could also create the files by making versions of the server which quit at various awkward places.) What this won't test so easily is recovery from failures during recovery. Regards, Hiram _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development _ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq: persistence implementation questions
I was only really suggesting that rather than test the entire JBoss server instance with a JBoss service, to focus on the individual component which performs this functionality. Assuming that the component can be started up from inside JUnit, then we can setup any given environment and given it a whirl. The same could be done by testing the entire server, but it would be more complicated and would not tell you that the given component will pass or fail the test (another component could be causing the component to fail... which is useful to know too). In this case, you could setup a log which might have 3 valid and then invalid garbage. JUnit could setup a log file which looked like this, then startup the component with that generated file and see if it performs as desired. Todo the same would mean knowing more about how the server starts up, its config and such to doctor up the environment to make the test work. Anyways, I am no QA/Unit Test expert... though work has kinda given me that hat to put on now (those bastards), so this is mostly specilation on my part. From last I played with jboss/testsuite (a while ago), most tests were run from JUnit by deploying something into a running server, pounding on it and then undeploying the app. I think this is good for many tests, there are some tests where we might want to avoid the deploy/rmi-exec/undeploy bits and simply execute the code directly. This would test the functionality of that code with out any additional factorys from the deploy/rmi/undeploy. In the case of JMS internals I think this is a great place to start. Could be that is what you are doing already... I would have to look at the testsuite again to become more familar with it. (1) that we are correctly flushing out our output to disk when we commit a transaction. All message are FULLY written to disk by the time the commit is issued. Probably the only way to ensure this would be to sync the FileDescriptor, then re-read the log. Assuming that FileDescriptor.sync() will actually sync correctly or throw an exception, once you pass this method you are assured that the data written has actually flushed... more or less. Could be that the fs was really a ramdrive, or a raid-cache with no battery backup, but there is nothing you can do about that. Once you write one, sync. You can re-read the log and expect to have exactly one log, which will have the expected attributes and such. Anything else is a problem. (2) that we do not attempt to recover messages from transaction that did not commit. If the tx did not commit would it have been sync'd to disk? Not really sure how all that XA fluff works, so I have no clue how to test that. (3) that we can deal with garbage at the end of out transaction logs. Re-reading the tx log after a set of pre-descripted events should provide coverage of most of the problems. Would HARD Killing the server after a commit test (2)??? How would we implement the HARD kill (JNI)?? I have no clue as to the best way to test a total VM failure. Sure a JNI could call exit(), but then you would have to have some external process monitoring the test, and then some way to start up a new vm and communicate with it. Perhaps we need a bit of JUnit framework to do this? This could be used to do the end-to-end testing, where the testing vm would start up the JBoss vm (or more than one for distributed tests) and the could kill them as needed. Then you could setup any given environemnt, perhaps it was a standalone JBoss, or perhaps it was a specialed TestCase or TestSuite, which is rather volitle. Does such a JUnit modification/integration suite exist? --jason ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq: persistence implementation questions
Right, deep testing of service internals does not really belong in the current jbosstest unit tests. Those are for testing client oriented interaction with the server. Each module should have its own unit tests for internal testing. Transaction failure semantics and recovery is getting to in between ground. You could do as Jeff suggested and make use of mock objects to deploy different version of the mq and other JBoss services to isolate the testing in either unit testsuite. - Original Message - From: Jason Dillon [EMAIL PROTECTED] To: Hiram Chirino [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Thursday, November 15, 2001 4:29 PM Subject: Re: [JBoss-dev] jbossmq: persistence implementation questions I was only really suggesting that rather than test the entire JBoss server instance with a JBoss service, to focus on the individual component which performs this functionality. Assuming that the component can be started up from inside JUnit, then we can setup any given environment and given it a whirl. The same could be done by testing the entire server, but it would be more complicated and would not tell you that the given component will pass or fail the test (another component could be causing the component to fail... which is useful to know too). In this case, you could setup a log which might have 3 valid and then invalid garbage. JUnit could setup a log file which looked like this, then startup the component with that generated file and see if it performs as desired. Todo the same would mean knowing more about how the server starts up, its config and such to doctor up the environment to make the test work. Anyways, I am no QA/Unit Test expert... though work has kinda given me that hat to put on now (those bastards), so this is mostly specilation on my part. From last I played with jboss/testsuite (a while ago), most tests were run from JUnit by deploying something into a running server, pounding on it and then undeploying the app. I think this is good for many tests, there are some tests where we might want to avoid the deploy/rmi-exec/undeploy bits and simply execute the code directly. This would test the functionality of that code with out any additional factorys from the deploy/rmi/undeploy. In the case of JMS internals I think this is a great place to start. Could be that is what you are doing already... I would have to look at the testsuite again to become more familar with it. (1) that we are correctly flushing out our output to disk when we commit a transaction. All message are FULLY written to disk by the time the commit is issued. Probably the only way to ensure this would be to sync the FileDescriptor, then re-read the log. Assuming that FileDescriptor.sync() will actually sync correctly or throw an exception, once you pass this method you are assured that the data written has actually flushed... more or less. Could be that the fs was really a ramdrive, or a raid-cache with no battery backup, but there is nothing you can do about that. Once you write one, sync. You can re-read the log and expect to have exactly one log, which will have the expected attributes and such. Anything else is a problem. (2) that we do not attempt to recover messages from transaction that did not commit. If the tx did not commit would it have been sync'd to disk? Not really sure how all that XA fluff works, so I have no clue how to test that. (3) that we can deal with garbage at the end of out transaction logs. Re-reading the tx log after a set of pre-descripted events should provide coverage of most of the problems. Would HARD Killing the server after a commit test (2)??? How would we implement the HARD kill (JNI)?? I have no clue as to the best way to test a total VM failure. Sure a JNI could call exit(), but then you would have to have some external process monitoring the test, and then some way to start up a new vm and communicate with it. Perhaps we need a bit of JUnit framework to do this? This could be used to do the end-to-end testing, where the testing vm would start up the JBoss vm (or more than one for distributed tests) and the could kill them as needed. Then you could setup any given environemnt, perhaps it was a standalone JBoss, or perhaps it was a specialed TestCase or TestSuite, which is rather volitle. Does such a JUnit modification/integration suite exist? --jason ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq: persistence implementation questions
Can mockobjects be integrated with JUnit? --jason On Thu, 15 Nov 2001, Jeff Tulley wrote: Use mock objects. www.mockobjects.com To use the automated mock object creator may require modifying the code a little (creating some more interfaces), but you can simply use the concept. It is the only great way to test complicated failure cases, especially when there is a complicated state that has to be established. The idea is to have an object that you can tell it what state to be in - for instance, to fail whenever you call a certain routine, etc, and what kind of failure. It implements the same interface as the real object, so that you can use it interchangeably. There are some other concepts with mockobjects also, such as input or call validation - the object can keep track of how it was called and with what parameters and in what order. This can all be validated against your expectations. the mock objects people call it endotesting because you are testing from the inside, but without tainting your actual code with any knowledge of the testing framework. (Besides the need to create an interface based on the object, which is a sometimes a good thing anyway, though it does add one more file to maintain). Pretty good stuff... Jeff Tulley ([EMAIL PROTECTED]) (801)861-5322 Novell, Inc., the leading provider of Net services software. Loren Rosen [EMAIL PROTECTED] 11/15/01 3:27:44 PM Hiram Chirino wrote: Hi Loren! From: Loren Rosen [EMAIL PROTECTED] To: [EMAIL PROTECTED] [EMAIL PROTECTED] Subject: [JBoss-dev] jbossmq: persistance implementation questions Date: Mon, 12 Nov 2001 20:22:27 -0800 * Performance tuning/scalability: has any work been done to address potential bottlenecks for high message throughput, large numbers of queues or topics, etc.? Work has gone into making reading/writing from storage more efficient. I don't think we have profiled the system enough yet to find out where the bottle necks are. A good first step would be to measure the single producer/consumer latency, and tune that. Low latency is a important goal in its own right, and moreover if the CPU overhead is too high you won't even be able to keep the disk busy writing messages. I can go ahead and do this. * recovery robustness: recovery is a tricky thing since there's so many possible ways things can fail. What's been done to think about/test-for/defend-against all the possibilities? Basic recovery test have been succeeding. It's hard to automate failure testing. More code review would be good in this area to see if we are missing anything. I'll look through the code more closely. But recovery is testable. What you have to do is create some corrupt data files (probably by using a byte-level editor to munge some existing data files). Then you need some scripts to start up the server with those files -- which doesn't fit so well with a pure-Java Junit-based test suite. (You could also create the files by making versions of the server which quit at various awkward places.) What this won't test so easily is recovery from failures during recovery. Regards, Hiram _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development _ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] jbossmq: persistance implementation questions
I've been looking a bit at the jboss mq persistance code, and I have some questions. I'll state some of them from a user perspective, but my motivation is as a developer; that is, to address some of the 'problems' or 'limitations' these questions may uncover. * Different DBMS system support: I realize the current schema is pretty simple, but nevertheless I think it's inevitible that you're going to want tweaks for different database servers, for performance reasons if nothing else. * Performance tuning/scalability: has any work been done to address potential bottlenecks for high message throughput, large numbers of queues or topics, etc.? * recovery robustness: recovery is a tricky thing since there's so many possible ways things can fail. What's been done to think about/test-for/defend-against all the possibilities? * two-phase commit: Is two-phase commit (XA) supported? I didn't see anything explicitly in the code, but pehaps I haven't yet read it carefully enough. * architecture: I haven't thought deeply about the trade-offs, but at first look it seems odd to create a transaction log table in a rdms, given that they can do the logging for you behind the scenes. _ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq: persistence implementation questions
Hi Loren! From: Loren Rosen [EMAIL PROTECTED] To: [EMAIL PROTECTED] [EMAIL PROTECTED] Subject: [JBoss-dev] jbossmq: persistance implementation questions Date: Mon, 12 Nov 2001 20:22:27 -0800 I've been looking a bit at the jboss mq persistence code, and I have some questions. I'll state some of them from a user perspective, but my motivation is as a developer; that is, to address some of the 'problems' or 'limitations' these questions may uncover. * Different DBMS system support: I realize the current schema is pretty simple, but nevertheless I think it's inevitable that you're going to want tweaks for different database servers, for performance reasons if nothing else. I guess will will make the code more abstract as we start supporting more databases. If a DBMS is different enough from the basic implementation, we could always create a new Persistence Manager just for that case. * Performance tuning/scalability: has any work been done to address potential bottlenecks for high message throughput, large numbers of queues or topics, etc.? Work has gone into making reading/writing from storage more efficient. I don't think we have profiled the system enough yet to find out where the bottle necks are. * recovery robustness: recovery is a tricky thing since there's so many possible ways things can fail. What's been done to think about/test-for/defend-against all the possibilities? Basic recovery test have been succeeding. It's hard to automate failure testing. More code review would be good in this area to see if we are missing anything. * two-phase commit: Is two-phase commit (XA) supported? I didn't see anything explicitly in the code, but pehaps I haven't yet read it carefully enough. We do have some basic support for XA. In phase 1 we store all message activity to persistent storage. In the seconds phase we mark the transaction as committed. Kinda simple since in a messaging system, we have fewer problems that could cause the prepare to fail (we don't need to get any locks and such). * architecture: I haven't thought deeply about the trade-offs, but at first look it seems odd to create a transaction log table in a rdms, given that they can do the logging for you behind the scenes. Not really, if we were to tie down every JMS session (transaction context) to a database connection, we might run out of database connection quickly. That is why we manage the 2 phase commit and don't delegate that work to the database XAResource. All the work that is done on the database is not considered to be transacted so the transaction log is used to be able to rollback incomplete JMS transactions. Regards, Hiram _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq: persistance implementation questions
I looked too. On 2001.11.12 23:22:27 -0500 Loren Rosen wrote: I've been looking a bit at the jboss mq persistance code, and I have some questions. I'll state some of them from a user perspective, but my motivation is as a developer; that is, to address some of the 'problems' or 'limitations' these questions may uncover. * Different DBMS system support: I realize the current schema is pretty simple, but nevertheless I think it's inevitible that you're going to want tweaks for different database servers, for performance reasons if nothing else. * Performance tuning/scalability: has any work been done to address potential bottlenecks for high message throughput, large numbers of queues or topics, etc.? There's a recent start at a message cache. It appears to me to duplicate the pm, so I suspect there is room for a great deal of improvement here. * recovery robustness: recovery is a tricky thing since there's so many possible ways things can fail. What's been done to think about/test-for/defend-against all the possibilities? XAResource.recover is not implemented. I think little has been done. The jdbc pm doesn't even make an attempt to recover from any failure when there is an open transaction. * two-phase commit: Is two-phase commit (XA) supported? I didn't see anything explicitly in the code, but pehaps I haven't yet read it carefully enough. I also am unclear about this. I have a hard time convincing myself it works in the absence of good tests. * architecture: I haven't thought deeply about the trade-offs, but at first look it seems odd to create a transaction log table in a rdms, given that they can do the logging for you behind the scenes. Among the odd features of the jdbc pm is that it requires the db connections it uses to have autocommit = true. As far as I can tell this is undocumented. I did a little (very little) thinking about this and wondered if the following would be plausible: 1. modify PersistenceManager interface to expose an XAResource interface, used for all transaction control. 2. For jdbc pm, expose the XAResource from the (jca wrapped) XA db connection. For other pm, write whatever is appropriate. 3. Expose the XAResource from the PM through the XASession or the jms ra. This will allow 1 phase commit optimization for a transaction involving both the db and jms, at least if the transaction is entirely on the same server. (jms server on same vm as beans) 4. I'm not sure if it is necessary to register a synchronization with the tm to tell the stuff above the pm to flush cached results to the pm before commit starts. I suspect not but I'm not sure. Something like this seems to be called a caching manager in the jca spec. I'm unlikely to have time to implement any of these soon but I like thinking about them. Thanks David Jencks _ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq: persistence implementation questions
On 2001.11.13 00:13:54 -0500 Hiram Chirino wrote: Hi Loren! From: Loren Rosen [EMAIL PROTECTED] To: [EMAIL PROTECTED] [EMAIL PROTECTED] Subject: [JBoss-dev] jbossmq: persistance implementation questions Date: Mon, 12 Nov 2001 20:22:27 -0800 I've been looking a bit at the jboss mq persistence code, and I have some questions. I'll state some of them from a user perspective, but my motivation is as a developer; that is, to address some of the 'problems' or 'limitations' these questions may uncover. * Different DBMS system support: I realize the current schema is pretty simple, but nevertheless I think it's inevitable that you're going to want tweaks for different database servers, for performance reasons if nothing else. I guess will will make the code more abstract as we start supporting more databases. If a DBMS is different enough from the basic implementation, we could always create a new Persistence Manager just for that case. * Performance tuning/scalability: has any work been done to address potential bottlenecks for high message throughput, large numbers of queues or topics, etc.? Work has gone into making reading/writing from storage more efficient. I don't think we have profiled the system enough yet to find out where the bottle necks are. * recovery robustness: recovery is a tricky thing since there's so many possible ways things can fail. What's been done to think about/test-for/defend-against all the possibilities? Basic recovery test have been succeeding. It's hard to automate failure testing. More code review would be good in this area to see if we are missing anything. * two-phase commit: Is two-phase commit (XA) supported? I didn't see anything explicitly in the code, but pehaps I haven't yet read it carefully enough. We do have some basic support for XA. In phase 1 we store all message activity to persistent storage. In the seconds phase we mark the transaction as committed. Kinda simple since in a messaging system, we have fewer problems that could cause the prepare to fail (we don't need to get any locks and such). * architecture: I haven't thought deeply about the trade-offs, but at first look it seems odd to create a transaction log table in a rdms, given that they can do the logging for you behind the scenes. Not really, if we were to tie down every JMS session (transaction context) to a database connection, we might run out of database connection quickly. This is only a problem with a non-xa driver. With an xa driver, unlimited numbers of transactions can use the same connection serially. Anyone actually using transactions with jms is going to need a non-toy db also supporting xa, so I don't see this as a limitation except possibly for easy free testing in an all java environment. But then, right now the xa tests in the testsuite fail for lack of an xa db. The (xa capable) firebird jca-jdbc driver is probably nearly far enough along to work for this. thanks david jencks That is why we manage the 2 phase commit and don't delegate that work to the database XAResource. All the work that is done on the database is not considered to be transacted so the transaction log is used to be able to rollback incomplete JMS transactions. Regards, Hiram _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ JDBC PersistenceManager
What is the point of the ConnectionFactoryLoader as an entity that is coupled to the RARDeployer? I don't see the point of having trivial deployement of the RAR classes independent of the resource adapator configuration. For a standalone resource adaptor it is not so combersome to create a RAR and then configure an mbean, but for a RAR deployed as part of an ear this does not make sense to me. I think it would be more straightforward to configure the RAR deployment using a jboss-ra.xml descriptor as part of the RAR just as any other deployment unit is handled. - Original Message - From: David Jencks [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, November 02, 2001 8:08 PM Subject: Re: [JBoss-dev] JBossMQ JDBC PersistenceManager Well, don't confuse the rar and the connectionfactory. What I have on my machine, using the mbean references, is like this: a rar gets a RARDeployment mbean, which among other things lets you see the default properties and classes easily, but mostly allows you to depend on it. The ConnectionFactoryLoader has mbean references (dependencies) to the required RAR(Deployment) and the ConnectionManager(Factory) it uses. So, it will deploy as soon as the exact things it needs are present. No need for the artificial reference to RARDeployer. Do you think we should follow weblogic's lead and allow and encourage including a jboss-service.xml in a rar to allow configuring ConnectionFactory(Loaders) within the rar? It's certainly easy enough, but I have my doubts about the wisdom. I definitely want to preserve the ability to have ConnectionFactoryLoader configurations outside the rar. david jencks On 2001.11.02 20:45:09 -0500 Scott M Stark wrote: I'm not particularly keen on how we couple the deployment properties of a RAR to an MBean configuration AND a JMX notification. This makes the configuration for a RAR inconsistent with any other J2EE component deployment where this information is specified via a JBoss specific descriptor to build a self contained deployment unit. The configuration is also more convoluted as you have to specifiy the name of the RAR deployer MBean so that notifications are received. This really makes no sense to someone who just happens to have a RAR to deploy and hasn't been dragged through the JMX bus school of religion. I definitely want to change this for 3.0 and so I'll pickup what you have and finish the deployment layer for 3.0 in a couple of weeks when I finish the book. I'm not sure if I want to introduce such a change for the handling of RARs into the 2.4 branch. Scott Stark Chief Technology Officer JBoss Group, LLC ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ JDBC PersistenceManager
I guess I have no particular problem having the ability to put a jboss-service.xml file in a rar, as long as we keep the ability to configure ConnectionFactoryLoader mbeans in any *-service.xml file. The case with external configuration is crucial for jdbc databases and the required DefaultDS, and could be pretty useful if someone uses the same rar in many applications. On 2001.11.03 03:46:09 -0500 Scott M Stark wrote: What is the point of the ConnectionFactoryLoader as an entity that is coupled to the RARDeployer? None coupled to the RARDeployer, but coupled to the rar itself, it gives you the ability to easily add/remove more connection factories at any time after the rar is deployed, without duplicating the rar classes or disturbing allready functioning ConnectionFactories. I don't see the point of having trivial deployement of the RAR classes independent of the resource adapator configuration. For a standalone resource adaptor it is not so combersome to create a RAR and then configure an mbean, but for a RAR deployed as part of an ear this does not make sense to me. I think it would be more straightforward to configure the RAR deployment using a jboss-ra.xml descriptor as part of the RAR just as any other deployment unit is handled. Did someone add the rar in ear already? In this case, you can configure the CFL in a jboss-service.xml in the ear. BTW, what do you think of putting rar's deployed in ears on the application classloader rather than the system classloader? Thus allowing several apps to have incompatible versions of the same rar. Also btw, the change in 2.4 to put the rar's into the mlet classloader made them non-redeployable I believe. However this is better than inaccessible. david jencks - Original Message - From: David Jencks [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, November 02, 2001 8:08 PM Subject: Re: [JBoss-dev] JBossMQ JDBC PersistenceManager Well, don't confuse the rar and the connectionfactory. What I have on my machine, using the mbean references, is like this: a rar gets a RARDeployment mbean, which among other things lets you see the default properties and classes easily, but mostly allows you to depend on it. The ConnectionFactoryLoader has mbean references (dependencies) to the required RAR(Deployment) and the ConnectionManager(Factory) it uses. So, it will deploy as soon as the exact things it needs are present. No need for the artificial reference to RARDeployer. Do you think we should follow weblogic's lead and allow and encourage including a jboss-service.xml in a rar to allow configuring ConnectionFactory(Loaders) within the rar? It's certainly easy enough, but I have my doubts about the wisdom. I definitely want to preserve the ability to have ConnectionFactoryLoader configurations outside the rar. david jencks On 2001.11.02 20:45:09 -0500 Scott M Stark wrote: I'm not particularly keen on how we couple the deployment properties of a RAR to an MBean configuration AND a JMX notification. This makes the configuration for a RAR inconsistent with any other J2EE component deployment where this information is specified via a JBoss specific descriptor to build a self contained deployment unit. The configuration is also more convoluted as you have to specifiy the name of the RAR deployer MBean so that notifications are received. This really makes no sense to someone who just happens to have a RAR to deploy and hasn't been dragged through the JMX bus school of religion. I definitely want to change this for 3.0 and so I'll pickup what you have and finish the deployment layer for 3.0 in a couple of weeks when I finish the book. I'm not sure if I want to introduce such a change for the handling of RARs into the 2.4 branch. Scott Stark Chief Technology Officer JBoss Group, LLC ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ JDBC PersistenceManager
Here is the disconnect I still have with regard to RARs. For RARs we have to configure a ConnectionFactoryLoader mbean instance for a particular RAR. For every other J2EE component we have an mbean that handles many deployments and obtains the deployment unit specific properties from standard + JBoss specific deployment descriptors. With the current RAR mechanism I cannot deploy a RAR from a remote URL since the RAR is not a self describing deployment unit(2.4, I don't know about 3.0). Why? The only point to not including a JBoss specific deployment descriptor in the RAR is that this allows easier configuration. This is simply a tools/packaing issue. You can use an unarchived RAR and achieve the same effect by simply editing the jboss-ra.xml file within the RAR directory. Scott Stark Chief Technology Officer JBoss Group, LLC - Original Message - From: David Jencks [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Saturday, November 03, 2001 5:49 AM Subject: Re: [JBoss-dev] JBossMQ JDBC PersistenceManager I guess I have no particular problem having the ability to put a jboss-service.xml file in a rar, as long as we keep the ability to configure ConnectionFactoryLoader mbeans in any *-service.xml file. The case with external configuration is crucial for jdbc databases and the required DefaultDS, and could be pretty useful if someone uses the same rar in many applications. On 2001.11.03 03:46:09 -0500 Scott M Stark wrote: What is the point of the ConnectionFactoryLoader as an entity that is coupled to the RARDeployer? None coupled to the RARDeployer, but coupled to the rar itself, it gives you the ability to easily add/remove more connection factories at any time after the rar is deployed, without duplicating the rar classes or disturbing allready functioning ConnectionFactories. I don't see the point of having trivial deployement of the RAR classes independent of the resource adapator configuration. For a standalone resource adaptor it is not so combersome to create a RAR and then configure an mbean, but for a RAR deployed as part of an ear this does not make sense to me. I think it would be more straightforward to configure the RAR deployment using a jboss-ra.xml descriptor as part of the RAR just as any other deployment unit is handled. Did someone add the rar in ear already? In this case, you can configure the CFL in a jboss-service.xml in the ear. BTW, what do you think of putting rar's deployed in ears on the application classloader rather than the system classloader? Thus allowing several apps to have incompatible versions of the same rar. Also btw, the change in 2.4 to put the rar's into the mlet classloader made them non-redeployable I believe. However this is better than inaccessible. david jencks - Original Message - From: David Jencks [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, November 02, 2001 8:08 PM Subject: Re: [JBoss-dev] JBossMQ JDBC PersistenceManager Well, don't confuse the rar and the connectionfactory. What I have on my machine, using the mbean references, is like this: a rar gets a RARDeployment mbean, which among other things lets you see the default properties and classes easily, but mostly allows you to depend on it. The ConnectionFactoryLoader has mbean references (dependencies) to the required RAR(Deployment) and the ConnectionManager(Factory) it uses. So, it will deploy as soon as the exact things it needs are present. No need for the artificial reference to RARDeployer. Do you think we should follow weblogic's lead and allow and encourage including a jboss-service.xml in a rar to allow configuring ConnectionFactory(Loaders) within the rar? It's certainly easy enough, but I have my doubts about the wisdom. I definitely want to preserve the ability to have ConnectionFactoryLoader configurations outside the rar. david jencks On 2001.11.02 20:45:09 -0500 Scott M Stark wrote: I'm not particularly keen on how we couple the deployment properties of a RAR to an MBean configuration AND a JMX notification. This makes the configuration for a RAR inconsistent with any other J2EE component deployment where this information is specified via a JBoss specific descriptor to build a self contained deployment unit. The configuration is also more convoluted as you have to specifiy the name of the RAR deployer MBean so that notifications are received. This really makes no sense to someone who just happens to have a RAR to deploy and hasn't been dragged through the JMX bus school of religion. I definitely want to change this for 3.0 and so I'll pickup what you have and finish the deployment layer for 3.0 in a couple of weeks when I finish the book. I'm not sure if I want to introduce
[JBoss-dev] JBossMQ JDBC PersistenceManager
Hi, Hiram, The technique you mentioned should generally works for most MBeans. But for ConnectionFactoryLoader, it's a little different. The org.jboss.resource.ConnectionFactoryLoader uses some kind of a two step initialization. The connection specified in your XML file is created and bound to JNDI only if the jboss-jdbc.rar is fully deployed. The init logic in ConnectionFactoryLoader looks like this: initService: - register itself as a deployment notification listener for the resource adapter (jboss-jdbc.rar) [it's a little bit more abstract that this, but you get the idea] startService: - if RARDeployer is present (it should because that's an explicit depends tag in the xml file), ask RARDeployer if jboss-jdbc.rar has been deployed. - if YES, continue to load the connection factory and perform all necessary initialization including JNDI name registration. - if NO, wait until it is notified (recall initService). To our dependency checker (AutoDeployer), ConnectionFactoryLoader is already deployed even if it has not fully initialize its internal connection factory. So, the deploy process continues and a JNDI NameNotFoundException is thrown when jbossmq-service.xml is deployed. I have traced thru the initialization steps in: http://www.jboss.org/forums/thread.jsp?forum=51thread=3359 It looks similar to this in this particular case (in sequence) 1. deploy/lib/jboss-jdbc.rar (don't deploy... wait until RARDeployer is deployed) 2. deploy/j2eedeployment-service.xml (now RARDeployer is deployed) 3. deploy/postgres-service.xml (deploy ConnectionFactoryLoader but cannot initialize its internal connection factory) 4. deploy/jbossmq-service.xml (NameNotFoundException) 5. deploy/lib/jboss-jdbc.rar (now, the JDBC resource adapter is deployed, RARDeployer fires deployment notification event to ConnectionFactoryLoader's listener. The listener initializes the internal connection factory and performs JNDI registration but it's too late for jbossmq-service.xml) I think the problem can be fixed if we can re-visit jboss-jdbc.rar once the RARDeployer is ready. That is, moving Step 5 up to Step 3. This is my investigation so far. I cc jboss-development to see if someone has a better idea. :) And I don't know which (jboss-dev or spydermq) mailing list is more appropriate. Charles --- Hiram Chirino [EMAIL PROTECTED] wrote: The problem with JDBC is because the jboss-jdbc.rar is deployed AFTER jbossmq-service.xml. So, when the PersistenceManager starts up, it complains about NameNotBoundException (JNDI exception). To workaround this problem, you can first move jbossmq-service.xml out of the deploy directory. After JBoss successfully starts up, you put the file back into the deploy directory. The PersistenceManager should then be deployed properly. To solve that problem, you might want to look into adding the a depends tag to the jbossmq-service.xml file so that it waits for the JBOSS-SYSTEM:service=ConnectionFactoryLoader,name=DefaultDS I think this will delay jbossmq deployment until the DefaultDS is deployed. It would be WAY cool if we could get the jbossmq JDBC PM to work with the Hypersonic SQL database that is included with the default JBoss dist. What do you think?? Regards, Hiram I am using PostgreSQL which uses some wierd settings for Blob. Obviously, the JDBC PersistenceManager is not ready for this... So I cannot test if the actual functionality of the PersistenceManager. Since there doesn't seem to be a real demand for PostgreSQL persistence on JMS, I will try to get another DB and try it out. Charles --- Hiram Chirino [EMAIL PROTECTED] wrote: Cool.. I assume you have sourceforge ID. Send it to me, I'll have Mark F. get you RW access to the CVS. I forgot what the dependency problem was with JDBC. Could you refresh me?? Regards, Hiram From: Charles Chan [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Date: Fri, 2 Nov 2001 08:26:21 -0800 (PST) Hi, Hiram, I think I have found a solution to the port scanning problem: http://www.jboss.org/forums/thread.jsp?forum=48thread=3269 Please take a look. As for the JDBC problem, can we assume that the MBean dependency will be handled by someone else? Charles __ Do You Yahoo!? Find a job, post your resume. http://careers.yahoo.com _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp __ Do You Yahoo!? Find a job, post your resume. http://careers.yahoo.com _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp __ Do
Re: [JBoss-dev] JBossMQ JDBC PersistenceManager
Interesting situation. I have on my machine a reimplementation of the depends mechanism that changes the RARDeployment sequence so it does not rely on the notifications noted here. However, it eliminates the separate init and start steps of mbean startup. The only place this is used is setting up the queues and durable subscriptions for jboss mq. I got the rolling logged pm working this new way but not file or jdbc and I kind of ran out of energy for this. Anyone want to help? david jencks On 2001.11.02 18:45:55 -0500 Charles Chan wrote: Hi, Hiram, The technique you mentioned should generally works for most MBeans. But for ConnectionFactoryLoader, it's a little different. The org.jboss.resource.ConnectionFactoryLoader uses some kind of a two step initialization. The connection specified in your XML file is created and bound to JNDI only if the jboss-jdbc.rar is fully deployed. The init logic in ConnectionFactoryLoader looks like this: initService: - register itself as a deployment notification listener for the resource adapter (jboss-jdbc.rar) [it's a little bit more abstract that this, but you get the idea] startService: - if RARDeployer is present (it should because that's an explicit depends tag in the xml file), ask RARDeployer if jboss-jdbc.rar has been deployed. - if YES, continue to load the connection factory and perform all necessary initialization including JNDI name registration. - if NO, wait until it is notified (recall initService). To our dependency checker (AutoDeployer), ConnectionFactoryLoader is already deployed even if it has not fully initialize its internal connection factory. So, the deploy process continues and a JNDI NameNotFoundException is thrown when jbossmq-service.xml is deployed. I have traced thru the initialization steps in: http://www.jboss.org/forums/thread.jsp?forum=51thread=3359 It looks similar to this in this particular case (in sequence) 1. deploy/lib/jboss-jdbc.rar (don't deploy... wait until RARDeployer is deployed) 2. deploy/j2eedeployment-service.xml (now RARDeployer is deployed) 3. deploy/postgres-service.xml (deploy ConnectionFactoryLoader but cannot initialize its internal connection factory) 4. deploy/jbossmq-service.xml (NameNotFoundException) 5. deploy/lib/jboss-jdbc.rar (now, the JDBC resource adapter is deployed, RARDeployer fires deployment notification event to ConnectionFactoryLoader's listener. The listener initializes the internal connection factory and performs JNDI registration but it's too late for jbossmq-service.xml) I think the problem can be fixed if we can re-visit jboss-jdbc.rar once the RARDeployer is ready. That is, moving Step 5 up to Step 3. This is my investigation so far. I cc jboss-development to see if someone has a better idea. :) And I don't know which (jboss-dev or spydermq) mailing list is more appropriate. Charles --- Hiram Chirino [EMAIL PROTECTED] wrote: The problem with JDBC is because the jboss-jdbc.rar is deployed AFTER jbossmq-service.xml. So, when the PersistenceManager starts up, it complains about NameNotBoundException (JNDI exception). To workaround this problem, you can first move jbossmq-service.xml out of the deploy directory. After JBoss successfully starts up, you put the file back into the deploy directory. The PersistenceManager should then be deployed properly. To solve that problem, you might want to look into adding the a depends tag to the jbossmq-service.xml file so that it waits for the JBOSS-SYSTEM:service=ConnectionFactoryLoader,name=DefaultDS I think this will delay jbossmq deployment until the DefaultDS is deployed. It would be WAY cool if we could get the jbossmq JDBC PM to work with the Hypersonic SQL database that is included with the default JBoss dist. What do you think?? Regards, Hiram I am using PostgreSQL which uses some wierd settings for Blob. Obviously, the JDBC PersistenceManager is not ready for this... So I cannot test if the actual functionality of the PersistenceManager. Since there doesn't seem to be a real demand for PostgreSQL persistence on JMS, I will try to get another DB and try it out. Charles --- Hiram Chirino [EMAIL PROTECTED] wrote: Cool.. I assume you have sourceforge ID. Send it to me, I'll have Mark F. get you RW access to the CVS. I forgot what the dependency problem was with JDBC. Could you refresh me?? Regards, Hiram From: Charles Chan [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Date: Fri, 2 Nov 2001 08:26:21 -0800 (PST) Hi, Hiram, I think I have found a solution to the port scanning problem: http://www.jboss.org/forums/thread.jsp?forum=48thread=3269 Please take a look. As for the JDBC problem, can we assume
Re: [JBoss-dev] JBossMQ JDBC PersistenceManager
Well, don't confuse the rar and the connectionfactory. What I have on my machine, using the mbean references, is like this: a rar gets a RARDeployment mbean, which among other things lets you see the default properties and classes easily, but mostly allows you to depend on it. The ConnectionFactoryLoader has mbean references (dependencies) to the required RAR(Deployment) and the ConnectionManager(Factory) it uses. So, it will deploy as soon as the exact things it needs are present. No need for the artificial reference to RARDeployer. Do you think we should follow weblogic's lead and allow and encourage including a jboss-service.xml in a rar to allow configuring ConnectionFactory(Loaders) within the rar? It's certainly easy enough, but I have my doubts about the wisdom. I definitely want to preserve the ability to have ConnectionFactoryLoader configurations outside the rar. david jencks On 2001.11.02 20:45:09 -0500 Scott M Stark wrote: I'm not particularly keen on how we couple the deployment properties of a RAR to an MBean configuration AND a JMX notification. This makes the configuration for a RAR inconsistent with any other J2EE component deployment where this information is specified via a JBoss specific descriptor to build a self contained deployment unit. The configuration is also more convoluted as you have to specifiy the name of the RAR deployer MBean so that notifications are received. This really makes no sense to someone who just happens to have a RAR to deploy and hasn't been dragged through the JMX bus school of religion. I definitely want to change this for 3.0 and so I'll pickup what you have and finish the deployment layer for 3.0 in a couple of weeks when I finish the book. I'm not sure if I want to introduce such a change for the handling of RARs into the 2.4 branch. Scott Stark Chief Technology Officer JBoss Group, LLC - Original Message - From: David Jencks [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, November 02, 2001 4:43 PM Subject: Re: [JBoss-dev] JBossMQ JDBC PersistenceManager Interesting situation. I have on my machine a reimplementation of the depends mechanism that changes the RARDeployment sequence so it does not rely on the notifications noted here. However, it eliminates the separate init and start steps of mbean startup. The only place this is used is setting up the queues and durable subscriptions for jboss mq. I got the rolling logged pm working this new way but not file or jdbc and I kind of ran out of energy for this. Anyone want to help? david jencks On 2001.11.02 18:45:55 -0500 Charles Chan wrote: Hi, Hiram, The technique you mentioned should generally works for most MBeans. But for ConnectionFactoryLoader, it's a little different. The org.jboss.resource.ConnectionFactoryLoader uses some kind of a two step initialization. The connection specified in your XML file is created and bound to JNDI only if the jboss-jdbc.rar is fully deployed. The init logic in ConnectionFactoryLoader looks like this: initService: - register itself as a deployment notification listener for the resource adapter (jboss-jdbc.rar) [it's a little bit more abstract that this, but you get the idea] startService: - if RARDeployer is present (it should because that's an explicit depends tag in the xml file), ask RARDeployer if jboss-jdbc.rar has been deployed. - if YES, continue to load the connection factory and perform all necessary initialization including JNDI name registration. - if NO, wait until it is notified (recall initService). To our dependency checker (AutoDeployer), ConnectionFactoryLoader is already deployed even if it has not fully initialize its internal connection factory. So, the deploy process continues and a JNDI NameNotFoundException is thrown when jbossmq-service.xml is deployed. I have traced thru the initialization steps in: http://www.jboss.org/forums/thread.jsp?forum=51thread=3359 It looks similar to this in this particular case (in sequence) 1. deploy/lib/jboss-jdbc.rar (don't deploy... wait until RARDeployer is deployed) 2. deploy/j2eedeployment-service.xml (now RARDeployer is deployed) 3. deploy/postgres-service.xml (deploy ConnectionFactoryLoader but cannot initialize its internal connection factory) 4. deploy/jbossmq-service.xml (NameNotFoundException) 5. deploy/lib/jboss-jdbc.rar (now, the JDBC resource adapter is deployed, RARDeployer fires deployment notification event to ConnectionFactoryLoader's listener. The listener initializes the internal connection factory and performs JNDI registration but it's too late for jbossmq-service.xml) I think the problem can be fixed if we can re-visit jboss-jdbc.rar once the RARDeployer
JBoss threads on Linux (Was: Re: [JBoss-dev] JBossMQ thread model)
Hi, Tobias Frech wrote: There is a interesting thread on linux threads here: http://www.jboss.org/forums/thread.jsp?forum=52thread=2273 Yes, and it kind of exposes the confusion on this. A few years back I did a lot of Linux kernel hacking, so I guess I know a little about this. It is correct that all the different java threads share the same memory image. They also share all open file descriptors. But I am not sure if this can be called lightweight threads: In the kernel, each of these threads are scheduled independently, and the recalculation of the dynamic priority happens independently for each thread. Conpare this to lightweight threads that are usually scheduled in a simple round-robin way, under the control of the dynamic priority of the process. Changing the kernel to allow more processes is simple, as you only have to change a constant and recompile the kernel. If you change this, and have a lot of threads sharing the same memory image (like in the JBoss case), you may also have to lower the constant defining the thread stack size in linuxthreads (part of glibc), as each thread needs its own stack in the shared memory image, and you do not have room for a lot of stacks if each stack occupies 2 MB as default. JBoss seems to run fine with a 256 KB stack size. But with a lot of threads, you will see another problem: Threads are processes in Linux, and their dynamic priority (goodness) has to be recalculated in the kernel. The algorithm Linux is using for this is slow, but good. Good dynamic priority recalculation means that the overall system feels very responsive even under load, but the recalculation also eats processor cycles. To get around that problem, IBM has developed a patch to the Linux scheduler that enables faster (but not quite as good) dynamic priority recalculation. This patch was not accepted into the kernel, as it makes the system feel less responsive under normal conditions. But even with all this done, it is just a matter of starting enough threads before you start seeing the same problems again. And you should also see similar problems on other systems if you start enough threads. Anyway, (non-realtime) threads are IMHO nothing but a programmer's convenience, and should not be used if they can be easily avoided. In particular, on native-threads Java VMs, starting and stopping threads is very slow compared to other operations because it involves calls to the operating system. Best Regards, Ole Husgaard. ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ thread model
On 5 Okt, Ole Husgaard wrote: Hi, I hope it is OK that I forward your reply to the list. David Maplesden wrote: Sounds like a good idea. It didn't feel when writing the code that an extra thread per connection would be a problem. If it is for you then you will probably want to pool the communication threads as well, as there is at least one of these per connection for the commonly used OIL and UIL communication mechanisms. Yes, I have been looking at this too, but found no easy solution. Maybe because I am not very familiar with I/O under Java. My problem is multiplexing several file stream reads on a single thread. Here, the Java libraries seem to lack features similar to the BSD select(2) syscall or the SysV poll(2) syscall. While something could be emulated with the InputStream.available() method, it would be a busy wait for input since this method would have to be called almost constantly. I believe that the new I/O system in JDK1.4 supports this, but we cannot require users to use that (yet). See JSR-51. There might be a way around that. One of the members in JSR-51 has written a non blocking io library for java (on Unix plattforms) that works with earlier releases of the JDK - nbio. According to the website, it is used in SwiftMQ to provide superior scalability and performance for MQ-based applications. Something to beat for JBossMQ ;-) //Peter Best Regards, Ole Husgaard. ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development //Peter -- Jobba hos oss: http://www.tim.se/weblab Peter Antman Technology in Media, Box 34105 100 26 Stockholm Systems ArchitectWWW: http://www.tim.se Email: [EMAIL PROTECTED]WWW: http://www.backsource.org Phone: +46-(0)8-506 381 11 Mobile: 070-675 3942 ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] JBossMQ thread model
The home page of NBIO is: http://www.cs.berkeley.edu/~mdw/proj/java-nbio/ From my reading of this in the past this only supported systems that support standard *NIX system calls. Maybe this would work on Win2k with Cygwin. Scott Stark Chief Technology Officer JBoss Group, LLC - Original Message - From: Peter Antman [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, October 05, 2001 5:44 AM Subject: Re: [JBoss-dev] JBossMQ thread model On 5 Okt, Ole Husgaard wrote: Hi, I hope it is OK that I forward your reply to the list. David Maplesden wrote: Sounds like a good idea. It didn't feel when writing the code that an extra thread per connection would be a problem. If it is for you then you will probably want to pool the communication threads as well, as there is at least one of these per connection for the commonly used OIL and UIL communication mechanisms. Yes, I have been looking at this too, but found no easy solution. Maybe because I am not very familiar with I/O under Java. My problem is multiplexing several file stream reads on a single thread. Here, the Java libraries seem to lack features similar to the BSD select(2) syscall or the SysV poll(2) syscall. While something could be emulated with the InputStream.available() method, it would be a busy wait for input since this method would have to be called almost constantly. I believe that the new I/O system in JDK1.4 supports this, but we cannot require users to use that (yet). See JSR-51. There might be a way around that. One of the members in JSR-51 has written a non blocking io library for java (on Unix plattforms) that works with earlier releases of the JDK - nbio. According to the website, it is used in SwiftMQ to provide superior scalability and performance for MQ-based applications. Something to beat for JBossMQ ;-) //Peter ___ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
