Tom, Did you set "DEBUG=False" in your settings.py file? If you don't Django will keep a record of all you sql statements called. That may be causing your out of memory error. -Alexei
Director of Web Development - Farallon Geographics, Inc. - 971.227.3173 On Fri, Oct 4, 2019 at 1:18 AM Shewring, Tom < [email protected]> wrote: > Hi, > > > > Thank you for you your replies. I took a snapshot of the server before I > started the upgrade process. I will roll back to the snapshot and follow > the process Cyrus has detailed. Before I do so, would you be able to > comment on the process I followed and the error I encountered - > > 1) Upgrade Yarn and Node.js to 1.13.0 and 10.15.2 respectively (this > brings the versions of this software up to the same level as the our other > working 4.4.1 environments) > > 2) python -m pip install --upgrade pip > > 3) pip install arches==4.4.1 --upgrade --no-binary :all: (this > completed successfully, upgrading directly from 4.1.1) > > 4) python manage.py migrate – (errors encountered) – is this due to > a lack of memory on the server? – (see final error massage - > django.db.utils.DatabaseError: out of memory for query result) > > > > Operations to perform: > > Apply all migrations: admin, auth, contenttypes, guardian, models, > oauth2_provider, sessions > > Running migrations: > > Applying models.2293_recent_edits... OK > > Applying models.2974_provisionaledits_in_editlog... OK > > Applying models.3098_custom_map_markers... OK > > Applying models.3199_graphmodel_color... OK > > Applying models.3201_remove_node_and_nodetype_branches... OK > > Applying models.3210_card_components... OK > > Applying models.3314_graphmodel_jsonldcontext... OK > > Applying models.3201_replace_node_and_nodetype_branches... OK > > Applying models.3201_second_removal_of_node_nodetype_branch... OK > > Applying models.deserialize_provisional_edits...Traceback (most recent > call last): > > File "manage.py", line 29, in <module> > > execute_from_command_line(sys.argv) > > File > "c:\obap\env4.1.1\lib\site-packages\django\core\management\__init__.py", > line 364, in execute_from_command_line > > utility.execute() > > File > "c:\obap\env4.1.1\lib\site-packages\django\core\management\__init__.py", > line 356, in execute > > self.fetch_command(subcommand).run_from_argv(self.argv) > > File > "c:\obap\env4.1.1\lib\site-packages\django\core\management\base.py", line > 283, in run_from_argv > > self.execute(*args, **cmd_options) > > File > "c:\obap\env4.1.1\lib\site-packages\django\core\management\base.py", line > 330, in execute > > output = self.handle(*args, **options) > > File > "c:\obap\env4.1.1\lib\site-packages\django\core\management\commands\migrate.py", > line 204, in handle > > fake_initial=fake_initial, > > File > "c:\obap\env4.1.1\lib\site-packages\django\db\migrations\executor.py", line > 115, in migrate > > state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, > fake_initial=fake_initial) > > File > "c:\obap\env4.1.1\lib\site-packages\django\db\migrations\executor.py", line > 145, in _migrate_all_forwards > > state = self.apply_migration(state, migration, fake=fake, > fake_initial=fake_initial) > > File > "c:\obap\env4.1.1\lib\site-packages\django\db\migrations\executor.py", line > 244, in apply_migration > > state = migration.apply(state, schema_editor) > > File > "c:\obap\env4.1.1\lib\site-packages\django\db\migrations\migration.py", > line 129, in apply > > operation.database_forwards(self.app_label, schema_editor, old_state, > project_state) > > File > "c:\obap\env4.1.1\lib\site-packages\django\db\migrations\operations\special.py", > line 193, in database_forwards > > self.code(from_state.apps, schema_editor) > > File > "c:\obap\env4.1.1\lib\site-packages\arches\app\models\migrations\deserialize_provisional_edits.py", > line 13, in forwards_func > > for tile in tiles: > > File "c:\obap\env4.1.1\lib\site-packages\django\db\models\query.py", > line 250, in __iter__ > > self._fetch_all() > > File "c:\obap\env4.1.1\lib\site-packages\django\db\models\query.py", > line 1118, in _fetch_all > > self._result_cache = list(self._iterable_class(self)) > > File "c:\obap\env4.1.1\lib\site-packages\django\db\models\query.py", > line 53, in __iter__ > > results = compiler.execute_sql(chunked_fetch=self.chunked_fetch) > > File > "c:\obap\env4.1.1\lib\site-packages\django\db\models\sql\compiler.py", line > 899, in execute_sql > > raise original_exception > > django.db.utils.DatabaseError: out of memory for query result > > > > Thank you > > > > Tom Shewring > > GIS Systems Developer > > IMT Development Centre > > 07748 182134 > > > > IMT Development and Programmes: Together, we enable the celebration of > England’s historic environment by empowering our customers and the public > through technology and innovation. > > > > *From:* Cyrus Hiatt [mailto:[email protected]] > *Sent:* 04 October 2019 01:28 > *To:* Adam Cox > *Cc:* Alexei Peters; Shewring, Tom; Arches Project > *Subject:* Re: [Arches] Upgrading to a specific version of Arches - 4.4.1 > (not the latest 4.4.2 version) > > > > Hi Tom - > > > > One of the challenges for upgrading from 4.1.1 will be upgrading your > project(s). This is because at version 4.2 we switched from Bower to Yarn > for js dependencies because bower was getting deprecated. As Adam has > mentioned, each version of Arches has the upgrade process outlined in the > release notes: > https://github.com/archesproject/arches/tree/master/releases. You could > go from version to version following the upgrade instructions for each > version. Alternatively, you could try the process below which is a bit > shorter. I test the upgrade from version to version with each release, so > it should go smoothly. I haven't tested the shortcut process below, but I > think it should be fine. In either case, there are quite a few steps going > from 4.1 to 4.4, so I would recommend creating a database backup and web > server snapshot (if possible) before getting started. > > > > *1. upgrade to version 4.3.1:* > > > > pip install arches==4.3.1 --upgrade --no-binary :all: > > pip uninstall pycryptodome, rdflib-jsonld > > pip install pycryptodome django-oauth-toolkit==1.1.2 PyLD[requests]==1.0.3 > pyprind==2.11.2 > > > > python manage.py migrate > > python manage.py es delete_indexes > > python manage.py es setup_indexes > > python manage.py es index_database > > > > *2. follow the project upgrade steps in the 4.2.0 release notes starting > with the yarn install:* > > > > > https://github.com/archesproject/arches/blob/master/releases/4.2.0.md#installing-yarn > > > > *3. Then follow the project upgrade steps for version 4.3.1:* > > > > > https://github.com/archesproject/arches/blob/master/releases/4.3.1.md#upgrading-an-arches-project > > > > *4. Upgrade to version 4.4.1:* > > > > pip install arches==4.4.1 --upgrade --no-binary :all: > > python manage.py migrate > > python manage.py es delete_indexes > > python manage.py es setup_indexes > > python manage.py es index_database > > > > *5. Then complete the project upgrades for 4.4.1:* > > > > > https://github.com/archesproject/arches/blob/master/releases/4.4.1.md#upgrading-an-arches-project > > > > Hope that helps and that your upgrade goes smoothly. Let me know if you > have any questions. > > > > - Cyrus > > > > > > > > On Thu, Oct 3, 2019 at 9:41 AM Adam Cox <[email protected]> wrote: > > Hi Tom, one other thought, sometimes I like to just create a brand new > virtual environment and install the new version in there, leaving the > existing on as is. Then you can switch which environment is used by your > app in your apache config (if that's how you're serving it), etc, and you > can switch back if something goes wrong with the pip installation process > (which, frankly, is unlikely). > > > > As for other actions to take, it's possible you will need to run > migrations in your project. I'd recommend looking back through release > notes for the incremental versions between 4.1.1 and 4.4.1, which can be > found on this forum. > > > > Adam > > > > On Thu, Oct 3, 2019 at 11:19 AM Alexei Peters <[email protected]> wrote: > > Hi Tom, > > The pip command to install a specific version should be this `pip install > myPackage=={version num} > > So for Arches that should be: > > > > ‘pip install arches==4.4.1 --upgrade --no-binary :all:’ > > > > Cheers, > > Alexei > > Director of Web Development - Farallon Geographics, Inc. - 971.227.3173 > > > > > > On Thu, Oct 3, 2019 at 5:36 AM Shewring, Tom < > [email protected]> wrote: > > Hi, > > > > We have an Arches deployment at 4.1.1, I wish to upgrade this environment > to *4.4.1* (not the latest 4.4.2 release). > > > > 1) Can you advise on the specific command to do this since ‘pip > install arches --upgrade --no-binary :all:’ will install 4.4.2 > > 2) Are there any other steps that I will need to take when upgrading > from 4.1.1 to 4.4.1 > > > > Many thanks > > > > Tom Shewring > > GIS Systems Developer > > IMT Development Centre > > Historic England > > > > [image: Historic England Logo] <http://www.historicengland.org.uk/> > > We are the public body that helps people care for, enjoy and celebrate > England's spectacular historic environment, from beaches and battlefields > to parks and pie shops. > Follow us: Facebook <https://www.facebook.com/HistoricEngland> | > Twitter <https://twitter.com/HistoricEngland> | Instagram > <https://www.instagram.com/historicengland/> Sign up to our newsletter > <http://webmail.historicenglandservices.org.uk/k/Historic-England/historic_england_preference_centre> > > > This e-mail (and any attachments) is confidential and may contain personal > views which are not the views of Historic England unless specifically > stated. If you have received it in error, please delete it from your system > and notify the sender immediately. Do not use, copy or disclose the > information in any way nor act in reliance on it. Any information sent to > Historic England may become publicly available. We respect your privacy and > the use of your information. Please read our full privacy policy > <https://www.historicengland.org.uk/terms/privacy-cookies/> for more > information. > > > > -- > -- To post, send email to [email protected]. To unsubscribe, > send email to [email protected]. For more > information, visit https://groups.google.com/d/forum/archesproject?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Arches Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/archesproject/4236dfe5e1b54308b93b8ba2d6e7c9cc%40SVMEX02.english-heritage.org.uk > <https://groups.google.com/d/msgid/archesproject/4236dfe5e1b54308b93b8ba2d6e7c9cc%40SVMEX02.english-heritage.org.uk?utm_medium=email&utm_source=footer> > . > > -- > -- To post, send email to [email protected]. To unsubscribe, > send email to [email protected]. For more > information, visit https://groups.google.com/d/forum/archesproject?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Arches Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/archesproject/CA%2BZLqy8oUEOsJoaPMWpip-ac3woMbTMCFxzRhg-PAdE6qoCA-g%40mail.gmail.com > <https://groups.google.com/d/msgid/archesproject/CA%2BZLqy8oUEOsJoaPMWpip-ac3woMbTMCFxzRhg-PAdE6qoCA-g%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- > -- To post, send email to [email protected]. To unsubscribe, > send email to [email protected]. For more > information, visit https://groups.google.com/d/forum/archesproject?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Arches Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/archesproject/CAGYBTavsKPzuTQcZyQR7RGhRV0UFH7FKRrGu_zTJG1QaqHuYAg%40mail.gmail.com > <https://groups.google.com/d/msgid/archesproject/CAGYBTavsKPzuTQcZyQR7RGhRV0UFH7FKRrGu_zTJG1QaqHuYAg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- -- To post, send email to [email protected]. To unsubscribe, send email to [email protected]. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/archesproject/CA%2BZLqy8Mm%3Dmu6p940i7wZd_krKPjui826FRT80v0SqWHfLAc1A%40mail.gmail.com.
