JupyterLab through Nginx Proxy Running on an EC2 Instance
April 19, 2019
While building something out for a client I had to host a JupyterLab instance on an EC2 instance and serve it with nginx for some additional flexibility. I had a bunch of fun, and while it didn't take long it's worth writing down. There is a blog out there that was pretty helpful, I want to expand on it. For reference, here is the blog I found useful through a stackoverflow answer. Note there are some things the other blog says to do in the notebook configuration you'll probably want to add yourself. I wanted this example to be minimal to it's working state, so I left out extraneous configuration. In addition to both of these blogs, you'll want to have another service manage the JupyterLab server in case it crashes. I recommend Supervisor
.
First thing first, I needed a server. I used an Amazon's EC2 service to create a server on AWS. I created a t2.micro Ubuntu 18.04 server with x86 architecture. I created a security group for this instance in which I opened port 80 and 22. Port 80 so that web traffic could reach the server, and 22 so that I could ssh into it.
with my server in place, I could ssh into it with
ssh -i mypebfile.pem ubuntu@publicip
Once you're in the server's command line, Let's install nginx.
sudo apt update
sudo apt install nginx
systemctl status nginx # make sure its installed and running
You should now be able to visit your server in a browser and see the famous nginx default screen at http://youriphere
.
Let's get our jupyter lab server going now. As the ubuntu user, let's create a virtual environment and install everything under there to keep everything clean. I'll let the commands speak for themselves.
cd /home/ubuntu sudo apt-get install python3-venv python3 -m venv .venv source .venv/bin/activate pip install jupyterlab jupyter lab --generate-config
With the generated config, we need to enable remote access to jupyter lab.
vim /home/ubuntu/.jupyter/jupyter_notebook_config.py
Find the following line and set it to true.
c.NotebookApp.allow_remote_access = True
Protip - you can use esc
+ :set number
to turn on line numbers, and the line should exist at line 82 if it is like mine. If it's changed since the writing of this, I recommend using vim's search feature. esc
+ /remote
. Hit esc
and then n
if you need to go to the next occurence of the word. When you're done hit enter.
Save the file and quit (:wq
), and we're done with this part!
Now let's configure nginx, this is the most complicated part. Navigate to sites-available
in the nginx directory and add in a new configuration file for the jupyter lab server we intend to proxy. We'll then link this file in sites-enabled
which is by default included in nginx.conf
which is why this will work.
cd /etc/nginx/sites-available
Add the following block into a file, I called the file jupyterlab
but it doesn't really matter.
server { listen 80 default_server; listen [::]:80 default_server; # "The ngx_http_proxy_module module allows passing requests to another server." # http://nginx.org/en/docs/http/ngx_http_proxy_module.html # https://developer.mozilla.org/en-US/docs/Web/HTTP/Protocol_upgrade_mechanism#Common_uses_for_this_mechanism server_name _; location / { proxy_read_timeout 300s; proxy_pass http://127.0.0.1:8888; proxy_set_header Host $host; proxy_set_header X-Real-Ip $remote_addr; # most recent trusted IP proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # prevents proxy from anonymyzing user } location ~ /api/kernels/ { proxy_pass http://127.0.0.1:8888; proxy_set_header Host $host; proxy_http_version 1.1; # create the initial HTTP/1.1 session to upgrade proxy_set_header Upgrade "websocket"; # upgrade a HTTP connection for web sockets proxy_set_header Connection "Upgrade"; # "Upgrade" must be listed here since it is a hop-by-hop header proxy_read_timeout 86400; } location ~ /terminals/ { proxy_pass http://127.0.0.1:8000; proxy_set_header Host $host; proxy_http_version 1.1; proxy_set_header Upgrade "websocket"; proxy_set_header Connection "Upgrade"; proxy_read_timeout 86400; } }
and then link the file to sites-available
as we mentioned earlier.
sudo ln -s /etc/nginx/sites-available/jupyterlab /etc/nginx/sites-enabled/
Finally, let's reload nginx and start the server
sudo nginx -s reload # reload the nginx configuration to capture our changes cd /home/ubuntu source .venv/bin/activate jupyter lab --no-browser --port=8888
Copy the token from the ouput of the "jupyter lab --no-browser --port=8888
" command, it should look like this: http://localhost:8888/?token=1c40e5998bda0723a025239f5095352041a2053afad8096f
and navigate to the URL in the browser of your EC2 server.
E.g. In Chrome mine looked something like this:
http://my.public.ip.address/?token=1c40e5998bda0723a025239f5095352041a2053afad8096f
For the sake of completeness (and for my own reference 😄), here are the options I enabled when I did want to use a password to avoid pasting the token in the generated config.
c.NotebookApp.allow_remote_access = True c.NotebookApp.password = 'sha1:59923e27d906:9fa0836e0c58d9421567a7b602e462665beb434b' c.NotebookApp.port = 8888 # this is default, so not too necessary...
and to start the process in the background:
nohup
, short for "No Hangups" will tell the computer to run the process to run until it finishes, even if the user logs off.&
tells it to run the process in the computer.
which gives us... nohup jupyter notebook --no-browser --port=8888 &