Re: ob-python: import local package into a session
On Tue, Nov 24 2020, Jack Kamm wrote: > If you install the package using either "python setup.py develop", or > "pip install -e", then Python will install your code via symlinks > instead of copying, so then you don't have to worry about reinstalling > every time you make an edit. Hey, that's good to know, thanks. > To switch between venv's in emacs, I use pyvenv: > https://github.com/jorgenschaefer/pyvenv Yes, that's what I use, as well. -- Joost Kremers Life has its moments
Re: ob-python: import local package into a session
I have also been dissatisfied with the current options for making local python libraries accessible in certain org files. The amount of setup that is required outside the org file itself was too large, especially if you want someone else who is not intimately familiar with python to be able to use it. My old solution was to modify the PYTHONPATH environment variable for the whole Emacs process. However, after a bit of digging inspired by this thread I now have a solution that is entirely local: advise ~org-babel-execute:python~ to set a new PYTHONPATH via the ~process-environment~ dynamic variable and set that from a buffer local variable for the local additions to PYTHONPATH along with getenv PYTHONPATH. A working example below. Best! Tom #+name: orgstrap #+begin_src elisp :results none :noweb noexport (defvar-local local-python-path nil) (defun advise--obe-python-path (command args) (let ((process-environment (or (and local-python-path (cons (format "PYTHONPATH=%s" (concat local-python-path (getenv "PYTHONPATH"))) process-environment)) process-environment))) (apply command args))) (advice-add #'org-babel-execute:python :around #'advise--obe-python-path) (setq-local local-python-path (concat default-directory "code:")) #+end_src On Tue, Nov 24, 2020 at 9:26 AM Jack Kamm wrote: > > Joost Kremers writes: > > > I haven't really considered the option to install the utility functions as a > > package in the virtual environment, because I expect to change and develop > > those > > functions together with the rest of the project. If it were a separate > > package, > > I'd need to reinstall it every time I make changes to it, which will > > probably > > happen often. > > If you install the package using either "python setup.py develop", or > "pip install -e", then Python will install your code via symlinks > instead of copying, so then you don't have to worry about reinstalling > every time you make an edit. > > To switch between venv's in emacs, I use pyvenv: > https://github.com/jorgenschaefer/pyvenv >
Re: ob-python: import local package into a session
Joost Kremers writes: > I haven't really considered the option to install the utility functions as a > package in the virtual environment, because I expect to change and develop > those > functions together with the rest of the project. If it were a separate > package, > I'd need to reinstall it every time I make changes to it, which will probably > happen often. If you install the package using either "python setup.py develop", or "pip install -e", then Python will install your code via symlinks instead of copying, so then you don't have to worry about reinstalling every time you make an edit. To switch between venv's in emacs, I use pyvenv: https://github.com/jorgenschaefer/pyvenv
Re: ob-python: import local package into a session
On Tue, Nov 24 2020, Maxim Nikulin wrote: > 2. It seems that *recommended* and more flexible way is per-project > (per-version) virtual environments: venv in python3, similar thing were > called virtualenv in python2: > https://docs.python.org/3/tutorial/venv.html Maybe there is a convenient > way to choose and switch venv's directly from emacs. In simple cases > venv could be activated before starting emacs. Yes, I'm using virtual environments. Took me a while to get that figured out, though. Python-the-language is nice enough, but Python-the-ecosystem is quite a different thing... (Who said there should only be one way to do something?) I haven't really considered the option to install the utility functions as a package in the virtual environment, because I expect to change and develop those functions together with the rest of the project. If it were a separate package, I'd need to reinstall it every time I make changes to it, which will probably happen often. -- Joost Kremers Life has its moments
Re: ob-python: import local package into a session
2020-11-23 Joost Kremers wrote: I can include packages installed in `site-packages` and have them available in all code blocks. But is there a way to import my own packages into a session? In particular, packages I haven't installed system-wide? I do not have a recipe ready for using with org mode, and maybe you have already tried the following instruments: 1. Besides system-wide site-packages directory, there is similar ones for users, used by e.g. `pip install` (unless it is disabled accordingly to some recommendations). 2. It seems that *recommended* and more flexible way is per-project (per-version) virtual environments: venv in python3, similar thing were called virtualenv in python2: https://docs.python.org/3/tutorial/venv.html Maybe there is a convenient way to choose and switch venv's directly from emacs. In simple cases venv could be activated before starting emacs. 3. The Hitchhiker's Guide to Python have a couple of related sections. They might be a bit outdated but could be a source for convincing arguments to use venv's or for some usage patterns. https://docs.python-guide.org/dev/virtualenvs/ https://docs.python-guide.org/dev/pip-virtualenv/
Re: ob-python: import local package into a session
Hi Jack, On Mon, Nov 23 2020, Jack Kamm wrote: > This shouldn't be ob-python or even Emacs specific. You can test whether > things work by typing "python" in the terminal and attempting to import > your module. It all seems to depend on the exact directories, though, and other than modifying =sys.path= there doesn't seem to be a way to make Org, Python and me happy at the same time. :-) I wanted to create a few small, related projects that would share the utility functions, so I thought I'd put them all in separate subpackages of a single package. That works with `M-x run-python` and with an LSP server, but when I then put an Org file in its own subdirectory inside the package, I couldn't import the utils subpackage. I guess putting a =:dir= header arg might resolve that, but the Org manual says that =:dir= should not be used with =:exports both=, which I was also using. Anyway, thanks to you and to John. I think I have a better idea now how it all works and what my options are. > By the way, are you using IPython or vanilla Python? I recently > encountered an issue trying to import modules through a symlink in > IPython, whereas it worked perfectly fine in a vanilla Python session. I'm using IPython in `M-x run-python`, but vanilla Python for Org. There are no symlinks involved, so I guess it shouldn't matter. -- Joost Kremers Life has its moments
Re: ob-python: import local package into a session
Jack Kamm writes: > You need to make sure your module is either in the working directory you > started the Python session in, or in your PYTHONPATH, for example by > adjusting os.env["PYTHONPATH"] before attempting to import the module. Sorry, this was incorrect, you need to set PYTHONPATH before starting Python. The correct way to do it from within Python is to use sys.path, as John points out.
Re: ob-python: import local package into a session
Hi Joost, > What I'm trying to do is to import a Python file with a bunch of utility > functions into the ob-python session. I thought this might be possible if I'd > structure my code as a regular Python package, because that works if I want to > import my utility functions into another Python file. But it doesn't seem to > work for the ob-python session. You need to make sure your module is either in the working directory you started the Python session in, or in your PYTHONPATH, for example by adjusting os.env["PYTHONPATH"] before attempting to import the module. This shouldn't be ob-python or even Emacs specific. You can test whether things work by typing "python" in the terminal and attempting to import your module. By the way, are you using IPython or vanilla Python? I recently encountered an issue trying to import modules through a symlink in IPython, whereas it worked perfectly fine in a vanilla Python session. Jack
Re: ob-python: import local package into a session
I usually do that by either having the py file in the working directory of the session, in which case you simply import it. Alternatively something like this in python: import sys sys.path.insert(0, '/path/to/dir/with/myfile.py') import myfile you could also append to sys.path if that matters. John --- Professor John Kitchin Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 @johnkitchin http://kitchingroup.cheme.cmu.edu On Mon, Nov 23, 2020 at 10:49 AM Joost Kremers wrote: > Hi all, > > If I have an Org file with Python source blocks, I can run them in a > session > with the `:session` header arg. That way, I can include packages installed > in > `site-packages` and have them available in all code blocks. But is there a > way > to import my own packages into a session? In particular, packages I haven't > installed system-wide? > > What I'm trying to do is to import a Python file with a bunch of utility > functions into the ob-python session. I thought this might be possible if > I'd > structure my code as a regular Python package, because that works if I > want to > import my utility functions into another Python file. But it doesn't seem > to > work for the ob-python session. > > Is there a way to achieve this? I don't *have* to structure my utility > functions > as a Python package, so if there's another way of doing this, I'd be > interested > as well. > > TIA > > Joost > > > -- > Joost Kremers > Life has its moments > >
ob-python: import local package into a session
Hi all, If I have an Org file with Python source blocks, I can run them in a session with the `:session` header arg. That way, I can include packages installed in `site-packages` and have them available in all code blocks. But is there a way to import my own packages into a session? In particular, packages I haven't installed system-wide? What I'm trying to do is to import a Python file with a bunch of utility functions into the ob-python session. I thought this might be possible if I'd structure my code as a regular Python package, because that works if I want to import my utility functions into another Python file. But it doesn't seem to work for the ob-python session. Is there a way to achieve this? I don't *have* to structure my utility functions as a Python package, so if there's another way of doing this, I'd be interested as well. TIA Joost -- Joost Kremers Life has its moments
