Jupyter Notebook Kernels: How to Add, Change, Remove

Jupyter Notebook Kernels: How to Add, Change, Remove

Last updated:
Table of Contents

Examples were run on Ubuntu Linux

Add Virtualenv as Python Kernel

Replace your-venv with your virtualenv name

  • Activate the virtualenv

    $ source your-venv/bin/activate
    
  • Install jupyter in the virtualenv

    (your-venv)$ pip install jupyter
    
  • Add the virtualenv as a jupyter kernel

    (your-venv)$ ipython kernel install --name "local-venv" --user
    
  • You can now select the created kernel your-env when you start Jupyter:

added-virtual-env-to-jupyter The newly-added virtualenv will show up here
the next time you open jupyter

Add Spark Kernel

Prerequisites: Java and a local Spark installation

To add Spache Spark Use a Jupyter plugin called Apache Toree.

Install the plugin into jupyter (replace Spark path with your installation directory):

$ pip install jupyter 
$ pip install --upgrade toree
$ jupyter toree install --user --spark_home=/home/felipe/Downloads/spark-2.4.3-bin-hadoop2.7
$ jupyter notebook

adding-spark-kernel-jupyter Start jupyter with the toree plugin
and you'll be able to link jupyter to
an existing local spark installation

Add Scala Kernel

Updated 2021

Example: install Scala 2.12.11 kernel with almond version 0.10.0:

  • Download almond and scala libs

    (coursier is a scala tool used to install almond)

    $ curl -Lo coursier https://git.io/coursier-cli && chmod +x coursier
    

    (replace scala and almond versions if you need to)

    $ ./coursier bootstrap \
        -r jitpack \
        -i user -I user:sh.almond:scala-kernel-api_2.12.11:0.10.0 \
        sh.almond:scala-kernel_2.12.11:0.10.0 \
        -o almond
    
  • Install Scala kernel in Jupyter

    $ ./almond --install --id scala_2_12_11  --display-name "Scala 2.12.11"
    
  • Open Jupyter notebook and select Scala kernel

    $ jupyter notebook
    

add-scala-kernel-to-jupyter-notebook Create Scala notebooks just as you
would python ones

List kernels

Use jupyter kernelspec list

$ jupyter kernelspec list
Available kernels:
  global-tf-python-3    /home/felipe/.local/share/jupyter/kernels/global-tf-python-3
  local_venv2           /home/felipe/.local/share/jupyter/kernels/local_venv2
  python2               /home/felipe/.local/share/jupyter/kernels/python2
  python36              /home/felipe/.local/share/jupyter/kernels/python36
  scala                 /home/felipe/.local/share/jupyter/kernels/scala

Remove kernel

Use jupyter kernelspec remove <kernel-name>

$ jupyter kernelspec remove old_kernel
Kernel specs to remove:
  old_kernel            /home/felipe/.local/share/jupyter/kernels/old_kernel
Remove 1 kernel specs [y/N]: y
[RemoveKernelSpec] Removed /home/felipe/.local/share/jupyter/kernels/old_kernel

Change Kernel name

  • 1) Use $ jupyter kernelspec list to see the folder the kernel is located in

  • 2) In that folder, open up file kernel.json and edit option "display_name"


Troubleshooting

java.lang.NoClassDefFoundError: scala/App$class

You are probably trying to run Spark 3.0+ on an older Toree version.

Upgrade Toree to 0.5 or 0.6+