JupyterLab through Nginx Proxy Running on an EC2 Instance
April 19, 2019
While building something out for a client I had to host a JupyterLab instance on an EC2 instance and serve it with nginx for some additional flexibility. I had a bunch of fun, and while it didn't take long it's worth writing down. There is a blog out there that was pretty helpful, I want to expand on it. For reference, here is the blog I found useful through a stackoverflow answer. Note there are some things the other blog says to do in the notebook configuration you'll probably want to add yourself. I wanted this example to be minimal to it's working state, so I left out extraneous configuration. In addition to both of these blogs, you'll want to have another service manage the JupyterLab server in case it crashes. I recommend Supervisor
.
First thing first, I needed a server. I used an Amazon's EC2 service to create a server on AWS. I created a t2.micro Ubuntu 18.04 server with x86 architecture. I created a security group for this instance in which I opened port 80 and 22. Port 80 so that web traffic could reach the server, and 22 so that I could ssh into it.
with my server in place, I could ssh into it with
ssh -i mypebfile.pem ubuntu@publicip
Once you're in the server's command line, Let's install nginx.
sudo apt update
sudo apt install nginx
systemctl status nginx # make sure its installed and running
You should now be able to visit your server in a browser and see the famous nginx default screen at http://youriphere
.
Let's get our jupyter lab server going now. As the ubuntu user, let's create a virtual environment and install everything under there to keep everything clean. I'll let the commands speak for themselves.
cd /home/ubuntu
sudo apt-get install python3-venv
python3 -m venv .venv
source .venv/bin/activate
pip install jupyterlab
jupyter lab --generate-config
With the generated config, we need to enable remote access to jupyter lab.
vim /home/ubuntu/.jupyter/jupyter_notebook_config.py
Find the following line and set it to true.
c.NotebookApp.allow_remote_access = True
Protip - you can use esc
+ :set number
to turn on line numbers, and the line should exist at line 82 if it is like mine. If it's changed since the writing of this, I recommend using vim's search feature. esc
+ /remote
. Hit esc
and then n
if you need to go to the next occurence of the word. When you're done hit enter.
Save the file and quit (:wq
), and we're done with this part!
Now let's configure nginx, this is the most complicated part. Navigate to sites-available
in the nginx directory and add in a new configuration file for the jupyter lab server we intend to proxy. We'll then link this file in sites-enabled
which is by default included in nginx.conf
which is why this will work.
cd /etc/nginx/sites-available
Add the following block into a file, I called the file jupyterlab
but it doesn't really matter.
server {
listen 80 default_server;
listen [::]:80 default_server;
# "The ngx_http_proxy_module module allows passing requests to another server."
# http://nginx.org/en/docs/http/ngx_http_proxy_module.html
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Protocol_upgrade_mechanism#Common_uses_for_this_mechanism
server_name _;
location / {
proxy_read_timeout 300s;
proxy_pass http://127.0.0.1:8888;
proxy_set_header Host $host;
proxy_set_header X-Real-Ip $remote_addr; # most recent trusted IP
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # prevents proxy from anonymyzing user
}
location ~ /api/kernels/ {
proxy_pass http://127.0.0.1:8888;
proxy_set_header Host $host;
proxy_http_version 1.1; # create the initial HTTP/1.1 session to upgrade
proxy_set_header Upgrade "websocket"; # upgrade a HTTP connection for web sockets
proxy_set_header Connection "Upgrade"; # "Upgrade" must be listed here since it is a hop-by-hop header
proxy_read_timeout 86400;
}
location ~ /terminals/ {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_read_timeout 86400;
}
}
and then link the file to sites-available
as we mentioned earlier.
sudo ln -s /etc/nginx/sites-available/jupyterlab /etc/nginx/sites-enabled/
Finally, let's reload nginx and start the server
sudo nginx -s reload # reload the nginx configuration to capture our changes
cd /home/ubuntu
source .venv/bin/activate
jupyter lab --no-browser --port=8888
Copy the token from the ouput of the "jupyter lab --no-browser --port=8888
" command, it should look like this: http://localhost:8888/?token=1c40e5998bda0723a025239f5095352041a2053afad8096f
and navigate to the URL in the browser of your EC2 server.
E.g. In Chrome mine looked something like this:
http://my.public.ip.address/?token=1c40e5998bda0723a025239f5095352041a2053afad8096f
For the sake of completeness (and for my own reference 😄), here are the options I enabled when I did want to use a password to avoid pasting the token in the generated config.
c.NotebookApp.allow_remote_access = True
c.NotebookApp.password = 'sha1:59923e27d906:9fa0836e0c58d9421567a7b602e462665beb434b'
c.NotebookApp.port = 8888 # this is default, so not too necessary...
and to start the process in the background:
nohup
, short for "No Hangups" will tell the computer to run the process to run until it finishes, even if the user logs off.&
tells it to run the process in the computer.
which gives us... nohup jupyter notebook --no-browser --port=8888 &