Emacs python setup

Posted on Tue 01 January 2019 in posts

I got a new laptop recently and so set up Emacs on it for the first time. Since my emacs init.el file has gotten progressively messier over the past several years, I took the opportunity to think about the variety of python packages I use and set things up cleanly. The following is what I settled on; the steps involved weren't as clear as I had hoped, so I'm writing this all down in case I need to do it again.

Some notes on how I use Emacs and python before I start:

  • I use conda to manage my virtual environments. Most projects, but not all, have a corresponding environment.yml file in their repository. I installed miniconda at ~/miniconda3 (conda version 4.5.12).

  • I do the early parts of my development, the playing around with ideas before I start writing scripts, in a Jupyter Notebook. I also do the end of my analysis in them, creating plots and putting the parts together in them. I keep most of my functions in scripts, runnable from the command line so that I can submit them to NYU's computing cluster. Therefore, I don't need the full IDE experience; I don't need to run a shell session from within Emacs, and I prefer using Jupyter to ein, org-babel, or anything similar.

  • I primarily use el-get to manage my Emacs packages, falling back on use-package when an appropriate el-get recipe isn't available. My configurations for both of these are very basic (and I installed use-package using el-get).

Because I don't need the full IDE experience and want to use conda to manage my virtual environments, I cannot use Elpy, which combines a variety of python-related packages together with a minimum of setup difficulty (here's a guide to getting started with it). So I decided on using jedi.el (with the company backend) for auto-completion, the included python-mode (in Emacs 26.1) for basic syntax highlighting, conda.el for managing virtual environments, and flycheck (with pylint and flake8) for syntax- and style-checking. Wanting to use conda and the company backend meant things were slightly more complicated to set up.

First, conda.el. This was very straightforward to setup just following the instructions on the Github page. The following is the relevant block in my init.el:

(use-package conda
  :ensure t)
(require 'conda)
;; if you want auto-activation (see below for details), include:
(conda-env-autoactivate-mode t)
(custom-set-variables
 '(conda-anaconda-home "~/miniconda3"))

Next, company-mode and jedi.el, both of which I can install using el-get. (Note that, if you want to use jedi with the company backend, do not install the regular jedi.el: you should only install company-jedi, see this issue).

(setq my:el-get-packages
      '(company-mode))
(el-get-bundle elpa:jedi-core)
(el-get-bundle company-jedi :depends (company-mode))
(eval-after-load "company-jedi"
    '(setq jedi:server-command (list "~/miniconda3/envs/emacs-jedi/bin/python" jedi:server-script)))
(require 'company-jedi)
(el-get 'sync my:el-get-packages)

However, I can't use the automatic jedi:install-server command, and so need to do some manual set up. I mostly followed the instructions from Update 2 of this stackoverflow answer, with some changes:

  1. Create a conda environment (for current example the environment is named emacs-jedi) by doing: conda create -n emacs-jedi python

  2. Install the following python packages: jedi, sexpdata, epc. Only jedi is on conda, so I did the following: pip install sexpdata epc; conda install jedi.

  3. Install the jediepcserver. Navigate to the jedi-core install directory (probably ~/.emacs.d/el-get/jedi-core; it should contain jediepcserver.py), then run: python setup.py install (note the directory has to exist, so the (el-get-bundle elpa:jedi-core) needs to be run before this).

After doing the above, company-jedi should now be setup. The following are the relevant config blocks in my init.el

(add-hook 'conda-postactivate-hook 'jedi:stop-server)
(add-hook 'conda-postdeactivate-hook 'jedi:stop-server)

(defun my/python-mode-hook ()
  (add-to-list 'company-backends 'company-jedi))

(add-hook 'python-mode-hook 'my/python-mode-hook)

(add-hook 'python-mode-hook 'jedi:setup)
(setq jedi:complete-on-dot t)

Finally, install and set-up flycheck. This is relatively straightforward: flycheck can be installed using el-get:

(setq my:el-get-packages
      '(flycheck))
(el-get 'sync my:el-get-packages)

Now, install the pylint and flake8 executables, in the base conda environment:

conda activate base
pip install flake8 pylint

And tell flycheck where to find the executables (and add a couple extra lines of configuration):

(add-hook 'after-init-hook #'global-flycheck-mode)
(setq-default flycheck-emacs-lisp-load-path 'inherit)
(setq flycheck-flake8-maximum-line-length 99)
(setq flycheck-python-pylint-executable "~/miniconda3/bin/pylint")
(setq flycheck-python-flake8-executable "~/miniconda3/bin/flake8")

Once that's done, all you need to do is check things are set up. Open up any python file in Emacs and type C-u C-c ! v to see the status of flycheck and, if pylint and flake8 are deactivated, type C-u C-c ! x to activate them.

If everything works, flycheck will start underlining things in yellow and red to tell you that things are against the style guide or will raise errors. If you start to type something, jedi will suggest possible completions, and all of these will be based on the packages installed in the appropriate virtual environment.

All told, the following shows the relevant parts of my init file:

(setq my:el-get-packages
      '(company-mode
        flycheck))
(el-get-bundle elpa:jedi-core)
(el-get-bundle company-jedi :depends (company-mode))
(eval-after-load "company-jedi"
    '(setq jedi:server-command (list "~/miniconda3/envs/emacs-jedi/bin/python" jedi:server-script)))
(require 'company-jedi)
(el-get 'sync my:el-get-packages)

(use-package conda
  :ensure t)
(require 'conda)
;; if you want auto-activation (see below for details), include:
(conda-env-autoactivate-mode t)
(custom-set-variables
 '(conda-anaconda-home "~/miniconda3"))
(add-hook 'conda-postactivate-hook 'jedi:stop-server)
(add-hook 'conda-postdeactivate-hook 'jedi:stop-server)

(defun my/python-mode-hook ()
  (add-to-list 'company-backends 'company-jedi))

(add-hook 'python-mode-hook 'my/python-mode-hook)
(add-hook 'python-mode-hook 'jedi:setup)
(setq jedi:complete-on-dot t)

(add-hook 'after-init-hook #'global-flycheck-mode)
(setq-default flycheck-emacs-lisp-load-path 'inherit)
(setq flycheck-flake8-maximum-line-length 99)
(setq flycheck-python-pylint-executable "~/miniconda3/bin/pylint")
(setq flycheck-python-flake8-executable "~/miniconda3/bin/flake8")

emacs