Re: [I] [Bug]: DuckDB - if a table output fails, the destination duckdb database file remains locked and can no longer be used. (hop)
hansva closed issue #3510: [Bug]: DuckDB - if a table output fails, the destination duckdb database file remains locked and can no longer be used. URL: https://github.com/apache/hop/issues/3510 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug]: DuckDB - if a table output fails, the destination duckdb database file remains locked and can no longer be used. (hop)
usbrandon closed issue #3510: [Bug]: DuckDB - if a table output fails, the destination duckdb database file remains locked and can no longer be used. URL: https://github.com/apache/hop/issues/3510 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug]: DuckDB - if a table output fails, the destination duckdb database file remains locked and can no longer be used. (hop)
sramazzina commented on issue #3510: URL: https://github.com/apache/hop/issues/3510#issuecomment-1961448917 The DuckDB file seems to remain locked only when an exception is raised during the execution of the table output. I'm trying to understand what's the difference. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug]: DuckDB - if a table output fails, the destination duckdb database file remains locked and can no longer be used. (hop)
sramazzina commented on issue #3510: URL: https://github.com/apache/hop/issues/3510#issuecomment-1961276859 My bad, the idea of using transactional run configuration to solve the file locking problem seems not to be a valuable solution because the problem still remain. On the other side, the suggestion on using Read Only connections is valuable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug]: DuckDB - if a table output fails, the destination duckdb database file remains locked and can no longer be used. (hop)
sramazzina commented on issue #3510: URL: https://github.com/apache/hop/issues/3510#issuecomment-1961169059 Looking at what I detailed in previous message, to have things working we have 2 options that must be managed: 1) If we want to RW the DuckDB database we must only use a single connection. And that is solved by _using a pipeline run configuration that is defined as transactional_ because that is the only way to achive a single connection per pipeline 2) If we only want to Read the DuckDB database we can use a normal pipeline run configuration BUT we must open the connection in read only mode by adding the appropriate flag to connection configuration as shown in the image below.  This option will work immediately after the feature #3661 will be applied so that the parameters specified will be applied in the connection in the proper way -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug]: DuckDB - if a table output fails, the destination duckdb database file remains locked and can no longer be used. (hop)
sramazzina commented on issue #3510: URL: https://github.com/apache/hop/issues/3510#issuecomment-1959352544 Ciao @usbrandon the behavior seems to be correct and not related to any issue in Apache Hop, according to what we are reading in this thread [duckdb file being used by another process](https://github.com/dagster-io/dagster/discussions/18737). We need to perform further investigations to understand what we can do to overcome this limitations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
