[jira] Commented: (JCR-1050) Remove synchronization from JNDI data sources
[ https://issues.apache.org/jira/browse/JCR-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12524165 ] Padraic Hannon commented on JCR-1050: - I've actually been focusing on creating a CRX based version more than Jackrabbit and have been testing that. I will try get this back into jackrabbit asap. I think this relates to JCR-890 as well. Remove synchronization from JNDI data sources - Key: JCR-1050 URL: https://issues.apache.org/jira/browse/JCR-1050 Project: Jackrabbit Issue Type: Improvement Components: core Reporter: Padraic Hannon Attachments: DatasourcePersistenceManager.java, JNDI_Datasource_Changes.diff, OracleDatasourcePersistenceManager.java Using datasources one should be able to rely on the application server to manage PreparedStatement caches therefore pre-creating and holding onto the connection for long periods of time should not be needed. This relates to improvement JCR-313, however, that change did not address the benefits one could see in using an application server controlled datasource. Even if jackrabbit does aim to use an embedded database such a system could be configured to use datasources and could benefit from the removal of the synchronization. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1050) Remove synchronization from JNDI data sources
[ https://issues.apache.org/jira/browse/JCR-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520504 ] Stefan Guggisberg commented on JCR-1050: have you got any benchmark results that you could share with us? Remove synchronization from JNDI data sources - Key: JCR-1050 URL: https://issues.apache.org/jira/browse/JCR-1050 Project: Jackrabbit Issue Type: Improvement Components: core Reporter: Padraic Hannon Attachments: DatasourcePersistenceManager.java, JNDI_Datasource_Changes.diff, OracleDatasourcePersistenceManager.java Using datasources one should be able to rely on the application server to manage PreparedStatement caches therefore pre-creating and holding onto the connection for long periods of time should not be needed. This relates to improvement JCR-313, however, that change did not address the benefits one could see in using an application server controlled datasource. Even if jackrabbit does aim to use an embedded database such a system could be configured to use datasources and could benefit from the removal of the synchronization. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1050) Remove synchronization from JNDI data sources
[ https://issues.apache.org/jira/browse/JCR-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520257 ] Stefan Guggisberg commented on JCR-1050: discussion on the dev list: -- Forwarded message -- From: Thomas Mueller [EMAIL PROTECTED] Date: Aug 2, 2007 9:33 AM Subject: Re: [jira] Created: (JCR-1050) Remove synchronization from JNDI data sources To: dev@jackrabbit.apache.org Hi, I'm not sure if I understand this request for improvement. Using datasources So you suggest to use DataSource.getConnection(..) instead of DriverManager.getConnection(..)? How do you get / create the datasource object, using JNDI? What about embedded applications where JNDI is not available? one should be able to rely on the application server to manage PreparedStatement caches Do you suggest to create a new PreparedStatement for each request? therefore pre-creating and holding onto the connection for long periods of time should not be needed. Could you explain the advantages of 'not holding onto the connection'? I know that MySQL closes connections after 8 hours idle time, are there any other advantages? This relates to improvement JCR-313, however, that change did not address the benefits one could see in using an application server controlled datasource. What are those benefits? Even if jackrabbit does aim to use an embedded database such a system could be configured to use datasources and could benefit from the removal of the synchronization. In what way would removal of the synchronization be a benefit? Do you think it would be faster without synchronization? How would you make sure statements are executed in the right order? Thanks, Thomas Remove synchronization from JNDI data sources - Key: JCR-1050 URL: https://issues.apache.org/jira/browse/JCR-1050 Project: Jackrabbit Issue Type: Improvement Components: core Reporter: Padraic Hannon Attachments: JNDI_Datasource_Changes.diff Using datasources one should be able to rely on the application server to manage PreparedStatement caches therefore pre-creating and holding onto the connection for long periods of time should not be needed. This relates to improvement JCR-313, however, that change did not address the benefits one could see in using an application server controlled datasource. Even if jackrabbit does aim to use an embedded database such a system could be configured to use datasources and could benefit from the removal of the synchronization. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1050) Remove synchronization from JNDI data sources
[ https://issues.apache.org/jira/browse/JCR-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520258 ] Stefan Guggisberg commented on JCR-1050: discussion on the dev list: -- Forwarded message -- From: hannonpi [EMAIL PROTECTED] Date: Aug 8, 2007 2:20 AM Subject: Re: [jira] Created: (JCR-1050) Remove synchronization from JNDI data sources To: dev@jackrabbit.apache.org See reply threaded below. Perhaps this should be moved into the Jira ticket? -paddy Thomas Mueller-6 wrote: Hi, I'm not sure if I understand this request for improvement. Using datasources So you suggest to use DataSource.getConnection(..) instead of DriverManager.getConnection(..)? How do you get / create the datasource object, using JNDI? What about embedded applications where JNDI is not available? response I attached code to the ticket. Basically, this assumes that one is running inside an application server container. I am not suggesting this be the only driver, just that the JNDI drive should be built in such a way as to make use of the facilities provided by JEE containers (datasources, jta, etc). /response one should be able to rely on the application server to manage PreparedStatement caches Do you suggest to create a new PreparedStatement for each request? response Yes, let the datasource or DB handle caching the PreparedStatements rather than holding them in an internal map. /response therefore pre-creating and holding onto the connection for long periods of time should not be needed. Could you explain the advantages of 'not holding onto the connection'? I know that MySQL closes connections after 8 hours idle time, are there any other advantages? response Why hold onto resources one is not using? Let other threads take them. /response This relates to improvement JCR-313, however, that change did not address the benefits one could see in using an application server controlled datasource. What are those benefits? response Less code in jackrabbit for managing transactions and less synchronization leading to less potential threading conflicts. /response Even if jackrabbit does aim to use an embedded database such a system could be configured to use datasources and could benefit from the removal of the synchronization. In what way would removal of the synchronization be a benefit? Do you think it would be faster without synchronization? How would you make sure statements are executed in the right order? response Our experience over the last year or so of using CQ and CRX has lead us to believe that synchronization has serious performance penalties in high traffic situations. In general I would think that the fewer synchronized parts the better. This is not a request to entirely do away with synchronized blocks. However, looking at the DB drivers it seemed that the sole purpose of such blocks was to handle the fact that statements and connections where held open for long periods by the driver. I would assume that allowing multiple threads to read would have serious performance implications and that allowing the container and db to manage transactions one could decide on the transaction isolation level outside of the core code to deal with dirty reads etc. /response Thanks, Thomas -- View this message in context: http://www.nabble.com/-jira--Created%3A-%28JCR-1050%29-Remove-synchronization-from-JNDI-data-sources-tf4203578.html#a12044986 Sent from the Jackrabbit - Dev mailing list archive at Nabble.com. Remove synchronization from JNDI data sources - Key: JCR-1050 URL: https://issues.apache.org/jira/browse/JCR-1050 Project: Jackrabbit Issue Type: Improvement Components: core Reporter: Padraic Hannon Attachments: JNDI_Datasource_Changes.diff Using datasources one should be able to rely on the application server to manage PreparedStatement caches therefore pre-creating and holding onto the connection for long periods of time should not be needed. This relates to improvement JCR-313, however, that change did not address the benefits one could see in using an application server controlled datasource. Even if jackrabbit does aim to use an embedded database such a system could be configured to use datasources and could benefit from the removal of the synchronization. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1050) Remove synchronization from JNDI data sources
[ https://issues.apache.org/jira/browse/JCR-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520259 ] Stefan Guggisberg commented on JCR-1050: discussion on the dev list: -- Forwarded message -- From: Thomas Mueller [EMAIL PROTECTED] Date: Aug 10, 2007 8:56 AM Subject: Re: [jira] Created: (JCR-1050) Remove synchronization from JNDI data sources To: dev@jackrabbit.apache.org Hi, I am not suggesting this be the only driver, just that the JNDI drive should be built in such a way as to make use of the facilities provided by JEE containers (datasources, jta, etc). I think using JNDI as an alternative way to get the connection is fine. Do you suggest to create a new PreparedStatement for each request? response Yes, let the datasource or DB handle caching the PreparedStatements rather than holding them in an internal map. /response I don't think there are advantages in using prepared statements from a data source compared to using your own prepared statements. pre-creating ... should not be needed. I agree, it's not required to create all prepared statements when connecting. It would be OK if they are created when required (and then put in a hash map or so). holding onto the connection for long periods ... should not be needed. Except for MySQL (where the connection drops after a few hours) I don't see a problem doing that. There is a risk (for all remote databases) that the connection drops temporarily (network cable disconnected or so), but if you want to solve that you need to add some reconnect functionality - even when using data sources. advantages of 'not holding onto the connection'? Why hold onto resources one is not using? Let other threads take them. You mean other threads inside Jackrabbit? As far as I know, the persistence engine of Jackrabbit doesn't require multiple connections. Or do you mean other threads inside other applications? I suggest not to access Jackrabbit databases directly. Less code in jackrabbit for managing transactions I don't think it would be less code. You anyway need to maintain the current behavior (using DriverManager to get the connection). So adding separate persistence managers (would be required for all databases) would double the maintenance work? I think there are already too many persistence managers. But I agree, getting the connection from a data source would make sense. This could be integrated into the current persistence manager(s). and less synchronization leading to less potential threading conflicts. You probably mean higher concurrency. However I don't think that this would be possible just because data sources are used. synchronization has serious performance penalties in high traffic situations. In general I would think that the fewer synchronized parts the better. When using one connection: Some JDBC drivers are not thread-safe, that means there is a risk accessing the same connection using multiple threads at the same time. Others are thread-safe, but synchronize internally, so there would be no benefit. When using multiple connections, there are new problems. Are you suggesting to use multiple connections inside one persistence manager? The connection defines the scope of the transaction, so using multiple connections would mean multiple concurrent transactions. As far as I know, the current Jackrabbit engine does not support this. Actually, I think Jackrabbit _should_ use one database connection per session. The problem is, the architecture is currently no like that. the purpose of synchronized blocks was to handle the fact that statements and connections where held open for long periods by the driver. I don't think this is the reason why synchronization is used (but I might be wrong). In my view, synchronization is used to make sure the JDBC objects (statements, result sets) are not accessed concurrently. that allowing multiple threads to read would have serious performance implications With the current architecture, I don't think removing synchronization would improve the performance. But if it does improve performance, or course this should be implemented. Thomas Remove synchronization from JNDI data sources - Key: JCR-1050 URL: https://issues.apache.org/jira/browse/JCR-1050 Project: Jackrabbit Issue Type: Improvement Components: core Reporter: Padraic Hannon Attachments: JNDI_Datasource_Changes.diff Using datasources one should be able to rely on the application server to manage PreparedStatement caches therefore pre-creating and holding onto the connection for long periods of time should not be needed. This relates to improvement JCR-313, however, that change did not address the benefits one could see in using an application server controlled
[jira] Commented: (JCR-1050) Remove synchronization from JNDI data sources
[
https://issues.apache.org/jira/browse/JCR-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520275
]
Stefan Guggisberg commented on JCR-1050:
paddy,
thanks for the patch, i appreciate your efforts.
i have a few general comments regarding the patch:
- the patch is *huge* (2k lines) and incorporates massive refactoring related
and other changes in several jackrabbit classes; this makes tracking and
understanding the actual changes very difficult at best.
- the subject of this issue suggests that only JNDI datasource related classes
would be affected ('Remove synchronization from JNDI data sources'). the scope
of the patch is much broader as far as i can tell from browsing through the
diff.
- the patch is incomplete; i wasn't able to apply it because of some missing
files.
rather than refactoring the current implementations i'd like to encourage you
to
write a separate, independant persistence manager (accepting some code
redundancy). that would enable us to perform one-to-one performance, functional
scalability tests. the test results would provide a better basis for
decision-making.
without such tests we can only guess and make assumptions.
cheers
stefan
Remove synchronization from JNDI data sources
-
Key: JCR-1050
URL: https://issues.apache.org/jira/browse/JCR-1050
Project: Jackrabbit
Issue Type: Improvement
Components: core
Reporter: Padraic Hannon
Attachments: JNDI_Datasource_Changes.diff
Using datasources one should be able to rely on the application server to
manage PreparedStatement caches therefore pre-creating and holding onto the
connection for long periods of time should not be needed. This relates to
improvement JCR-313, however, that change did not address the benefits one
could see in using an application server controlled datasource. Even if
jackrabbit does aim to use an embedded database such a system could be
configured to use datasources and could benefit from the removal of the
synchronization.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (JCR-1050) Remove synchronization from JNDI data sources
Stefan Guggisberg (JIRA) wrote: [ https://issues.apache.org/jira/browse/JCR-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520257 ] Stefan Guggisberg commented on JCR-1050: discussion on the dev list: -- Forwarded message -- From: Thomas Mueller [EMAIL PROTECTED] Date: Aug 2, 2007 9:33 AM Subject: Re: [jira] Created: (JCR-1050) Remove synchronization from JNDI data sources To: dev@jackrabbit.apache.org Hi, I'm not sure if I understand this request for improvement. Using datasources So you suggest to use DataSource.getConnection(..) instead of DriverManager.getConnection(..)? How do you get / create the datasource object, using JNDI? What about embedded applications where JNDI is not available? I really would like to see Jackrabbit to support DataSource and JNDI. This simplifies the usage in an application server and corporate environments (corporate = the AS admins configure the datasource in the AS and will ask questions why you got a JEE app which can not use the Jdbc Pool for connections ... no chance that in your role as a 'application provider' you will the the production DB password!). How about - Use commons-dbcp for creating and managing datasource - All DB backed PM/FS only use an 'injected' DataSource to get a single connection for now. This greatly reduces the redundant create-connection-from-driver-manager logic from FS, PM and for all implementation types (bundled, simple, ...). Reconnects fetch a fresh connection from the data source. - Create a JNDI PM/FS wrapper for datasource based PM/FS which would fetch the data source from JNDI and inject it into the wrapped PM/FS. one should be able to rely on the application server to manage PreparedStatement caches Do you suggest to create a new PreparedStatement for each request? As already mentioned before in this thread: a JEE datasource pool handles PrepStat caching nicely (nice article: http://www.theserverside.com/tt/articles/article.tss?l=Prepared-Statments) I'm not sure if commons-dbcp would do that, too ... ??? therefore pre-creating and holding onto the connection for long periods of time should not be needed. Could you explain the advantages of 'not holding onto the connection'? I know that MySQL closes connections after 8 hours idle time, are there any other advantages? The mysql idle timeout can be configured on the server side. Also, some firewalls close idle connections. Connection pools can 'health' check the connections before handing one to the application (eg JR). Most DB vendors provide optimized health checking utils (eg for mysql when configuring a datasource on JBoss). This relates to improvement JCR-313, however, that change did not address the benefits one could see in using an application server controlled datasource. What are those benefits? Even if jackrabbit does aim to use an embedded database such a system could be configured to use datasources and could benefit from the removal of the synchronization. In what way would removal of the synchronization be a benefit? Do you think it would be faster without synchronization? How would you make sure statements are executed in the right order? Thanks, Thomas Remove synchronization from JNDI data sources - Key: JCR-1050 URL: https://issues.apache.org/jira/browse/JCR-1050 Project: Jackrabbit Issue Type: Improvement Components: core Reporter: Padraic Hannon Attachments: JNDI_Datasource_Changes.diff Using datasources one should be able to rely on the application server to manage PreparedStatement caches therefore pre-creating and holding onto the connection for long periods of time should not be needed. This relates to improvement JCR-313, however, that change did not address the benefits one could see in using an application server controlled datasource. Even if jackrabbit does aim to use an embedded database such a system could be configured to use datasources and could benefit from the removal of the synchronization.
[jira] Commented: (JCR-1050) Remove synchronization from JNDI data sources
[ https://issues.apache.org/jira/browse/JCR-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520309 ] Padraic Hannon commented on JCR-1050: - That makes sense, I was trying to eliminate duplication of code and ensure that there was a common code base. I will do a more coarse implementation first so we can get a better idea of what the changes are. -Paddy Remove synchronization from JNDI data sources - Key: JCR-1050 URL: https://issues.apache.org/jira/browse/JCR-1050 Project: Jackrabbit Issue Type: Improvement Components: core Reporter: Padraic Hannon Attachments: JNDI_Datasource_Changes.diff Using datasources one should be able to rely on the application server to manage PreparedStatement caches therefore pre-creating and holding onto the connection for long periods of time should not be needed. This relates to improvement JCR-313, however, that change did not address the benefits one could see in using an application server controlled datasource. Even if jackrabbit does aim to use an embedded database such a system could be configured to use datasources and could benefit from the removal of the synchronization. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1050) Remove synchronization from JNDI data sources
[ https://issues.apache.org/jira/browse/JCR-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517999 ] Padraic Hannon commented on JCR-1050: - One note on the diff, I am sure there are defects, associated with these files. I am working through the tests now to ensure that I get them all. Remove synchronization from JNDI data sources - Key: JCR-1050 URL: https://issues.apache.org/jira/browse/JCR-1050 Project: Jackrabbit Issue Type: Improvement Components: core Affects Versions: 0.9, 1.0, 1.0.1, 1.1, 1.1.1, 1.2.1, 1.2.2, 1.2.3, 1.3, 1.3.1, 1.4, 2.0 Reporter: Padraic Hannon Attachments: JNDI_Datasource_Changes.diff Using datasources one should be able to rely on the application server to manage PreparedStatement caches therefore pre-creating and holding onto the connection for long periods of time should not be needed. This relates to improvement JCR-313, however, that change did not address the benefits one could see in using an application server controlled datasource. Even if jackrabbit does aim to use an embedded database such a system could be configured to use datasources and could benefit from the removal of the synchronization. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1050) Remove synchronization from JNDI data sources
[ https://issues.apache.org/jira/browse/JCR-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517311 ] Padraic Hannon commented on JCR-1050: - Yeah I saw that in the code. However, from what I can tell the only place where the connection autocommit is explicitly turned off and rollback() and commit() are called is in the method store(ChangeLog). Assuming that one is running in a container and has access to a datasource I would also assume that one has access to a UserTransaction object. If that is the case, rather than using a connection's transaction handling one can delegate the transaction handling to jta. Remove synchronization from JNDI data sources - Key: JCR-1050 URL: https://issues.apache.org/jira/browse/JCR-1050 Project: Jackrabbit Issue Type: Improvement Components: core Affects Versions: 0.9, 1.0, 1.0.1, 1.1, 1.1.1, 1.2.1, 1.2.2, 1.2.3, 1.3, 1.3.1, 1.4, 2.0 Reporter: Padraic Hannon Using datasources one should be able to rely on the application server to manage PreparedStatement caches therefore pre-creating and holding onto the connection for long periods of time should not be needed. This relates to improvement JCR-313, however, that change did not address the benefits one could see in using an application server controlled datasource. Even if jackrabbit does aim to use an embedded database such a system could be configured to use datasources and could benefit from the removal of the synchronization. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
