Sudhanva Narayana
Sudhanva Narayana

Follow

Sudhanva Narayana

Follow
Install JupyterHub on AWS Elastic Kubernetes Service (EKS)

Install JupyterHub on AWS Elastic Kubernetes Service (EKS)

Sudhanva Narayana's photo
Sudhanva Narayana
·Mar 2, 2021·

3 min read

Play this article

Table of contents

  • Introduction
  • GPU enabled Docker image
  • Guidelines For a new installation
  • Creating the Cluster

Introduction

GPU enabled Docker image

  • Use the image from https://hub.docker.com/r/cschranz/gpu-jupyter for GPUs to be recognised inside your environment (it's already in the config file, just choose that as the environment)
  • Other tensorflow images won't detect the GPUs due to issues with CUDA and cuDNN

Guidelines For a new installation

Initial Setup

https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html

  • Create a new group called admin
  • Assign Administrator Access policy
  • Create user, enable programmatic and AWS console access, and add to the admin group (if required)
  • MFA in the root account must be enabled. Assign Virtual MFA (Microsoft Authenticator)
  • Root account access key should be deleted
  • Create additional users and groups as necessary
  • Sign out of root account
  • Sign in as i-am-user
  • Cloud Formation Role needed to create any of the following components. This also needs IAM access

Creating a VPC

https://docs.aws.amazon.com/eks/latest/userguide/create-public-private-vpc.html

  • Add the necessary policies to the I-AM role.
  • Create a VPC from cloud formation using the CF template
  • Default is Public-Private

Creating the Cluster

https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html#modify-endpoint-access

  • Create cluster from the EKS console
  • Add security groups and subnets related to the previously created VPC
  • Security group has ControlPlaneSecurityGroup in the name available in the drop down
  • Configure cluster end point access. It is Public by default

Attaching Nodes to the Cluster

Opening a ticket on AWS: AWS does not let you create GPU instances, so a ticket must be created for this

Nodes do not connect to cluster: This is because of a changed configuration from AWS. MapPublicIpOnLaunch should be set to true in the VPC settings.

JupyterHub Configuration:

Previous issue

Browse all issues

Next issue

Did you find this article valuable?

Support Sudhanva Narayana by becoming a sponsor. Any amount is appreciated!

See recent sponsors Learn more about Hashnode Sponsors
 
Share this