Sudhanva Narayana
Sudhanva Narayana

Sudhanva Narayana

Install JupyterHub on AWS Elastic Kubernetes Service (EKS)

Install JupyterHub on AWS Elastic Kubernetes Service (EKS)

Sudhanva Narayana's photo
Sudhanva Narayana
·Mar 2, 2021·

3 min read

Subscribe to my newsletter and never miss my upcoming articles

Play this article

Table of contents

  • Introduction
  • GPU enabled Docker image
  • Guidelines For a new installation
  • Creating the Cluster

Introduction

GPU enabled Docker image

  • Use the image from https://hub.docker.com/r/cschranz/gpu-jupyter for GPUs to be recognised inside your environment (it's already in the config file, just choose that as the environment)
  • Other tensorflow images won't detect the GPUs due to issues with CUDA and cuDNN

Guidelines For a new installation

Initial Setup

https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html

  • Create a new group called admin
  • Assign Administrator Access policy
  • Create user, enable programmatic and AWS console access, and add to the admin group (if required)
  • MFA in the root account must be enabled. Assign Virtual MFA (Microsoft Authenticator)
  • Root account access key should be deleted
  • Create additional users and groups as necessary
  • Sign out of root account
  • Sign in as i-am-user
  • Cloud Formation Role needed to create any of the following components. This also needs IAM access

Creating a VPC

https://docs.aws.amazon.com/eks/latest/userguide/create-public-private-vpc.html

  • Add the necessary policies to the I-AM role.
  • Create a VPC from cloud formation using the CF template
  • Default is Public-Private

Creating the Cluster

https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html#modify-endpoint-access

  • Create cluster from the EKS console
  • Add security groups and subnets related to the previously created VPC
  • Security group has ControlPlaneSecurityGroup in the name available in the drop down
  • Configure cluster end point access. It is Public by default

Attaching Nodes to the Cluster

Opening a ticket on AWS: AWS does not let you create GPU instances, so a ticket must be created for this

Nodes do not connect to cluster: This is because of a changed configuration from AWS. MapPublicIpOnLaunch should be set to true in the VPC settings.

JupyterHub Configuration:

Previous issue

Browse all issues

Next issue

Did you find this article valuable?

Support Sudhanva Narayana by becoming a sponsor. Any amount is appreciated!

See recent sponsors Learn more about Hashnode Sponsors
 
Share this