your program runs completely (thanks), and I can't reproduce any problem.
changed line 1 as written to:
# Line 1.
# If nothing happens on line 1, line 3 fails.
insert_integer(0) # If I do this (insert-only) line 3 fails.
#database_action(0) # If I do this (query, insert) line 3 works.
and no issue. output is below. Send along a stack trace for more hints.
2022-08-22 19:16:02,322 INFO sqlalchemy.engine.Engine SELECT DATABASE()
2022-08-22 19:16:02,322 INFO sqlalchemy.engine.Engine [raw sql] ()
2022-08-22 19:16:02,322 INFO sqlalchemy.engine.Engine SELECT @@sql_mode
2022-08-22 19:16:02,322 INFO sqlalchemy.engine.Engine [raw sql] ()
2022-08-22 19:16:02,323 INFO sqlalchemy.engine.Engine SELECT
@@lower_case_table_names
2022-08-22 19:16:02,323 INFO sqlalchemy.engine.Engine [raw sql] ()
2022-08-22 19:16:02,323 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,324 INFO sqlalchemy.engine.Engine SELECT COUNT(*) FROM
information_schema.tables WHERE table_schema = %s AND table_name = %s
2022-08-22 19:16:02,324 INFO sqlalchemy.engine.Engine [generated in 0.00016s]
('test', 'mytable')
2022-08-22 19:16:02,325 INFO sqlalchemy.engine.Engine
DROP TABLE mytable
2022-08-22 19:16:02,325 INFO sqlalchemy.engine.Engine [no key 0.00012s] ()
2022-08-22 19:16:02,330 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,330 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,331 INFO sqlalchemy.engine.Engine SELECT COUNT(*) FROM
information_schema.tables WHERE table_schema = %s AND table_name = %s
2022-08-22 19:16:02,331 INFO sqlalchemy.engine.Engine [cached since 0.006941s
ago] ('test', 'mytable')
2022-08-22 19:16:02,332 INFO sqlalchemy.engine.Engine
CREATE TABLE mytable (
id INTEGER NOT NULL,
PRIMARY KEY (id)
)
2022-08-22 19:16:02,333 INFO sqlalchemy.engine.Engine [no key 0.00140s] ()
2022-08-22 19:16:02,342 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,344 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,345 INFO sqlalchemy.engine.Engine INSERT INTO mytable (id)
VALUES (%s)
2022-08-22 19:16:02,345 INFO sqlalchemy.engine.Engine [generated in 0.00021s]
(0,)
2022-08-22 19:16:02,346 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,362 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,362 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,365 INFO sqlalchemy.engine.Engine SELECT mytable.id
FROM mytable
WHERE mytable.id = %s
2022-08-22 19:16:02,365 INFO sqlalchemy.engine.Engine SELECT mytable.id
FROM mytable
WHERE mytable.id = %s
2022-08-22 19:16:02,365 INFO sqlalchemy.engine.Engine [generated in 0.00024s]
(1,)
2022-08-22 19:16:02,365 INFO sqlalchemy.engine.Engine [generated in 0.00024s]
(3,)
2022-08-22 19:16:02,366 INFO sqlalchemy.engine.Engine ROLLBACK
2022-08-22 19:16:02,366 INFO sqlalchemy.engine.Engine ROLLBACK
2022-08-22 19:16:02,367 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,367 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,367 INFO sqlalchemy.engine.Engine INSERT INTO mytable (id)
VALUES (%s)
2022-08-22 19:16:02,367 INFO sqlalchemy.engine.Engine [cached since 0.0218s
ago] (3,)
2022-08-22 19:16:02,367 INFO sqlalchemy.engine.Engine INSERT INTO mytable (id)
VALUES (%s)
2022-08-22 19:16:02,367 INFO sqlalchemy.engine.Engine [cached since 0.02189s
ago] (1,)
2022-08-22 19:16:02,367 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,367 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,368 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,369 INFO sqlalchemy.engine.Engine SELECT mytable.id
FROM mytable
WHERE mytable.id = %s
2022-08-22 19:16:02,369 INFO sqlalchemy.engine.Engine [cached since 0.003665s
ago] (4,)
2022-08-22 19:16:02,369 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,369 INFO sqlalchemy.engine.Engine SELECT mytable.id
FROM mytable
WHERE mytable.id = %s
2022-08-22 19:16:02,370 INFO sqlalchemy.engine.Engine [cached since 0.004593s
ago] (2,)
2022-08-22 19:16:02,370 INFO sqlalchemy.engine.Engine ROLLBACK
2022-08-22 19:16:02,370 INFO sqlalchemy.engine.Engine ROLLBACK
2022-08-22 19:16:02,370 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,370 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,370 INFO sqlalchemy.engine.Engine INSERT INTO mytable (id)
VALUES (%s)
2022-08-22 19:16:02,370 INFO sqlalchemy.engine.Engine [cached since 0.0254s
ago] (4,)
2022-08-22 19:16:02,371 INFO sqlalchemy.engine.Engine INSERT INTO mytable (id)
VALUES (%s)
2022-08-22 19:16:02,371 INFO sqlalchemy.engine.Engine [cached since 0.02555s
ago] (2,)
2022-08-22 19:16:02,371 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,371 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,372 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,372 INFO sqlalchemy.engine.Engine SELECT mytable.id
FROM mytable
WHERE mytable.id = %s
2022-08-22 19:16:02,372 INFO sqlalchemy.engine.Engine [cached since 0.007373s
ago] (5,)
2022-08-22 19:16:02,373 INFO sqlalchemy.engine.Engine ROLLBACK
2022-08-22 19:16:02,373 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,373 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,373 INFO sqlalchemy.engine.Engine SELECT mytable.id
FROM mytable
WHERE mytable.id = %s
2022-08-22 19:16:02,373 INFO sqlalchemy.engine.Engine [cached since 0.008211s
ago] (7,)
2022-08-22 19:16:02,373 INFO sqlalchemy.engine.Engine INSERT INTO mytable (id)
VALUES (%s)
2022-08-22 19:16:02,373 INFO sqlalchemy.engine.Engine [cached since 0.02819s
ago] (5,)
2022-08-22 19:16:02,374 INFO sqlalchemy.engine.Engine ROLLBACK
2022-08-22 19:16:02,374 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,374 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,374 INFO sqlalchemy.engine.Engine INSERT INTO mytable (id)
VALUES (%s)
2022-08-22 19:16:02,374 INFO sqlalchemy.engine.Engine [cached since 0.0291s
ago] (7,)
2022-08-22 19:16:02,375 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,375 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,375 INFO sqlalchemy.engine.Engine SELECT mytable.id
FROM mytable
WHERE mytable.id = %s
2022-08-22 19:16:02,375 INFO sqlalchemy.engine.Engine [cached since 0.01013s
ago] (6,)
2022-08-22 19:16:02,375 INFO sqlalchemy.engine.Engine ROLLBACK
2022-08-22 19:16:02,376 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,376 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,376 INFO sqlalchemy.engine.Engine INSERT INTO mytable (id)
VALUES (%s)
2022-08-22 19:16:02,376 INFO sqlalchemy.engine.Engine [cached since 0.03086s
ago] (6,)
2022-08-22 19:16:02,376 INFO sqlalchemy.engine.Engine SELECT mytable.id
FROM mytable
WHERE mytable.id = %s
2022-08-22 19:16:02,376 INFO sqlalchemy.engine.Engine [cached since 0.01102s
ago] (8,)
2022-08-22 19:16:02,376 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,377 INFO sqlalchemy.engine.Engine ROLLBACK
2022-08-22 19:16:02,377 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,377 INFO sqlalchemy.engine.Engine INSERT INTO mytable (id)
VALUES (%s)
2022-08-22 19:16:02,377 INFO sqlalchemy.engine.Engine [cached since 0.03192s
ago] (8,)
2022-08-22 19:16:02,377 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,378 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,378 INFO sqlalchemy.engine.Engine SELECT mytable.id
FROM mytable
WHERE mytable.id = %s
2022-08-22 19:16:02,378 INFO sqlalchemy.engine.Engine [cached since 0.01299s
ago] (9,)
2022-08-22 19:16:02,378 INFO sqlalchemy.engine.Engine ROLLBACK
2022-08-22 19:16:02,379 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,379 INFO sqlalchemy.engine.Engine INSERT INTO mytable (id)
VALUES (%s)
2022-08-22 19:16:02,379 INFO sqlalchemy.engine.Engine [cached since 0.03381s
ago] (9,)
2022-08-22 19:16:02,379 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,381 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,381 INFO sqlalchemy.engine.Engine SELECT mytable.id
FROM mytable
WHERE mytable.id = %s
2022-08-22 19:16:02,381 INFO sqlalchemy.engine.Engine [cached since 0.01605s
ago] (10,)
2022-08-22 19:16:02,381 INFO sqlalchemy.engine.Engine ROLLBACK
2022-08-22 19:16:02,382 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,382 INFO sqlalchemy.engine.Engine INSERT INTO mytable (id)
VALUES (%s)
2022-08-22 19:16:02,382 INFO sqlalchemy.engine.Engine [cached since 0.03694s
ago] (10,)
2022-08-22 19:16:02,383 INFO sqlalchemy.engine.Engine COMMIT
2022-08-22 19:16:02,387 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,388 INFO sqlalchemy.engine.Engine SELECT mytable.id
FROM mytable
WHERE mytable.id = %s
2022-08-22 19:16:02,388 INFO sqlalchemy.engine.Engine [generated in 0.00011s]
(20,)
2022-08-22 19:16:02,389 INFO sqlalchemy.engine.Engine ROLLBACK
2022-08-22 19:16:02,389 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2022-08-22 19:16:02,389 INFO sqlalchemy.engine.Engine INSERT INTO mytable (id)
VALUES (%s)
2022-08-22 19:16:02,389 INFO sqlalchemy.engine.Engine [cached since 0.04428s
ago] (20,)
2022-08-22 19:16:02,390 INFO sqlalchemy.engine.Engine COMMIT
On Mon, Aug 22, 2022, at 4:21 PM, Geert Jan Talens wrote:
> Hi,
>
> I am trying to perform some database actions after multiprocessing some
> database manipulations but in most cases I get:
> sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError)
> (2006, 'MySQL server has gone away')
>
> The code below is a working example of the problem I'm having, where the
> success of line 3 (in main) for some reason depends on what happens on line 1.
>
> Any help would be appreciated!
>
> Regards,
> Geert Jan
>
> from sqlalchemy import create_engine, select
> from sqlalchemy import Column, Integer
> from sqlalchemy.orm import declarative_base, sessionmaker
>
> import multiprocessing as mp
> import numpy as np
>
> Base = declarative_base()
> Session = sessionmaker()
>
>
> class MyTable(Base):
> """This table lists all observed frames."""
>
> __tablename__ = 'mytable'
>
> # Columns in the table.
> id = Column(Integer, primary_key=True, autoincrement=False)
>
>
> def insert_integer(integer):
>
> with Session() as session:
>
> record = MyTable(id=int(integer))
> session.add(record)
> session.commit()
>
> return
>
>
> def query_integer(integer):
>
> with Session() as session:
>
> statement = select(MyTable).where(MyTable.id == int(integer))
> result = session.execute(statement).scalar_one_or_none()
>
> return result
>
>
> def database_action(integer):
>
> result = query_integer(integer)
> if result is None:
> insert_integer(integer)
> else:
> print("integer already in database.")
>
> return
>
>
> def initializer(engine):
> # Doesn't matter if I use on or the other.
> engine.dispose(close=False)
> # engine.pool = engine.pool.recreate()
>
>
> def main():
>
> # Database setup.
> engine = create_engine("mysql+mysqldb://user:[email protected]/testdb")
> Session.configure(bind=engine)
> Base.metadata.drop_all(bind=engine)
> Base.metadata.create_all(bind=engine)
>
> # Line 1.
> # If nothing happens on line 1, line 3 fails.
> # insert_integer(0) # If I do this (insert-only) line 3 fails.
> database_action(0) # If I do this (query, insert) line 3 works.
>
> # Line 2.
> with mp.Pool(2, initializer, (engine,)) as pool:
> pool.map(database_action, np.arange(1, 11))
>
> # Line 3.
> database_action(20) # Fails if line 1 is insert_integer.
>
> return
>
>
> if __name__ == '__main__':
> main()
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/5597b069-3725-44de-88c4-4964d95d84cen%40googlegroups.com
>
> <https://groups.google.com/d/msgid/sqlalchemy/5597b069-3725-44de-88c4-4964d95d84cen%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/sqlalchemy/d4255029-5593-413d-a128-ee36505b46cd%40www.fastmail.com.
