JupyterLab through Nginx Proxy Running on an EC2 Instance

April 20, 2019, 2:11 a.m.


While building something out for a client I had to host a JupyterLab instance on an EC2 instance and serve it with nginx for some additional flexibility. I had a bunch of fun, and while it didn't take long it's worth writing down. There is a blog out there that was pretty helpful, I want to expand on it. For reference, here is the blog I found useful through a stackoverflow answer. Note there are some things the other blog says to do in the notebook configuration you'll probably want to add yourself. I wanted this example to be minimal to it's working state, so I left out extraneous configuration. In addition to both of these blogs, you'll want to have another service manage the JupyterLab server in case it crashes. I recommend Supervisor.

First thing first, I needed a server. I used an Amazon's EC2 service to create a server on AWS. I created a t2.micro Ubuntu 18.04 server with x86 architecture. I created a security group for this instance in which I opened port 80 and 22. Port 80 so that web traffic could reach the server, and 22 so that I could ssh into it.

with my server in place, I could ssh into it with

ssh -i mypebfile.pem ubuntu@publicip

Once you're in the server's command line, Let's install nginx.

sudo apt update
sudo apt install nginx
systemctl status nginx  # make sure its installed and running

You should now be able to visit your server in a browser and see the famous nginx default screen at http://youriphere.

Let's get our jupyter lab server going now. As the ubuntu user, let's create a virtual environment and install everything under there to keep everything clean. I'll let the commands speak for themselves.

cd /home/ubuntu
sudo apt-get install python3-venv
python3 -m venv .venv
source .venv/bin/activate
pip install jupyterlab
jupyter lab --generate-config

With the generated config, we need to enable remote access to jupyter lab.

vim /home/ubuntu/.jupyter/jupyter_notebook_config.py

Find the following line and set it to true.

c.NotebookApp.allow_remote_access = True

Protip - you can use esc + :set number to turn on line numbers, and the line should exist at line 82 if it is like mine. If it's changed since the writing of this, I recommend using vim's search feature. esc + /remote. Hit esc and then n if you need to go to the next occurence of the word. When you're done hit enter.

Save the file and quit (:wq), and we're done with this part!

Now let's configure nginx, this is the most complicated part. Navigate to sites-available in the nginx directory and add in a new configuration file for the jupyter lab server we intend to proxy. We'll then link this file in sites-enabled which is by default included in nginx.conf which is why this will work.

cd /etc/nginx/sites-available

Add the following block into a file, I called the file jupyterlab but it doesn't really matter.

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        # "The ngx_http_proxy_module module allows passing requests to another server."
        # http://nginx.org/en/docs/http/ngx_http_proxy_module.html
        # https://developer.mozilla.org/en-US/docs/Web/HTTP/Protocol_upgrade_mechanism#Common_uses_for_this_mechanism

        server_name _;

        location / {
                proxy_read_timeout 300s;
                proxy_pass http://127.0.0.1:8888;
                proxy_set_header Host $host;
                proxy_set_header X-Real-Ip $remote_addr;  # most recent trusted IP
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;  # prevents proxy from anonymyzing user
        }
        location ~ /api/kernels/ {
                proxy_pass http://127.0.0.1:8888;
                proxy_set_header Host $host;
                proxy_http_version 1.1;  # create the initial HTTP/1.1 session to upgrade
                proxy_set_header Upgrade "websocket";  # upgrade a HTTP connection for web sockets
                proxy_set_header Connection "Upgrade";  # "Upgrade" must be listed here since it is a hop-by-hop header
                proxy_read_timeout 86400;
        }
        location ~ /terminals/ {
                proxy_pass            http://127.0.0.1:8000;
                proxy_set_header      Host $host;
                proxy_http_version    1.1;
                proxy_set_header      Upgrade "websocket";
                proxy_set_header      Connection "Upgrade";
                proxy_read_timeout    86400;
        }
}

and then link the file to sites-available as we mentioned earlier.

sudo ln -s /etc/nginx/sites-available/jupyterlab /etc/nginx/sites-enabled/

Finally, let's reload nginx and start the server

sudo nginx -s reload  # reload the nginx configuration to capture our changes
cd /home/ubuntu
source .venv/bin/activate
jupyter lab --no-browser --port=8888

Copy the token from the ouput of the "jupyter lab --no-browser --port=8888" command, it should look like this: http://localhost:8888/?token=1c40e5998bda0723a025239f5095352041a2053afad8096f and navigate to the URL in the browser of your EC2 server.

E.g. In Chrome mine looked something like this:

http://my.public.ip.address/?token=1c40e5998bda0723a025239f5095352041a2053afad8096f

For the sake of completeness (and for my own reference 😄), here are the options I enabled when I did want to use a password to avoid pasting the token in the generated config.

c.NotebookApp.allow_remote_access = True
c.NotebookApp.password = 'sha1:59923e27d906:9fa0836e0c58d9421567a7b602e462665beb434b'
c.NotebookApp.port = 8888  # this is default, so not too necessary...

and to start the process in the background:

  • nohup, short for "No Hangups" will tell the computer to run the process to run until it finishes, even if the user logs off.
  • & tells it to run the process in the computer.

which gives us... nohup jupyter notebook --no-browser --port=8888 &

Comment Enter a new comment: