Internal Nublado controller API

The Nublado controller is built as a Docker image intended for deployment in a Kubernetes cluster using Phalanx and is not supported as an installed Python module. This documentation therefore exists only to assist developers and code analysis and does not define a supported API.

controller Package

The Nublado controller service.

Variables

__version__

The application version string (PEP 440 / SemVer compatible).

controller.background Module

Nublado controller background processing.

Classes

BackgroundTaskManager(*, image_service, ...)

Manage Nublado controller background tasks.

Class Inheritance Diagram

controller.config Module

Global configuration parsing.

Classes

BaseVolumeSource	Source of a volume to be mounted in the lab.
Config	Nublado controller configuration.
ContainerImage	Docker image that may be run as a container.
DisabledFileserverConfig	Configuration when user file servers are disabled.
EnabledFileserverConfig	Configuration for enabled user file servers.
FileserverConfig	Base configuration for user file servers.
HostPathVolumeSource	Path on Kubernetes node to mount in the container.
LabConfig	Configuration for spawning user labs.
LabInitContainer	A container to run as an init container before the user's lab.
LabNSSFiles	Rules for /etc/passwd and /etc/group inside the lab.
LabSecret	A secret to make available to lab containers.
LabSizeDefinition	Possible size of lab.
NFSVolumeSource	NFS volume to mount in the container.
PVCVolumeResources	Resources for a persistent volume claim.
PVCVolumeSource	A PVC to create to materialize the volume to mount in the container.
TmpSource(value[, names, module, qualname, ...])	Whether /tmp should come from disk (and thus ephemeralStorage) or memory-as-tmpfs (and thus pod memory).
UserHomeDirectorySchema(value[, names, ...])	Algorithm for building a user's home directory path.
VolumeConfig	A volume that may be mounted inside a container.
VolumeMountConfig	The mount of a volume inside a container.

Class Inheritance Diagram

controller.constants Module

Global constants.

Variables

ARGO_CD_ANNOTATIONS	Annotations to add to most created objects.
CONFIGURATION_PATH	Default path to controller configuration.
DOCKER_CREDENTIALS_PATH	Default path to the Docker API secrets.
DROPDOWN_SENTINEL_VALUE	Used in the lab form for image_list when image_dropdown is used.
GROUPNAME_REGEX	Regex matching all valid group names.
FILE_SERVER_RECONCILE_INTERVAL	How frequently to refresh file server state from Kubernetes.
IMAGE_REFRESH_INTERVAL	How frequently to refresh the list of remote and cached images.
KUBERNETES_NAME_PATTERN	Pattern matching valid Kubernetes names.
KUBERNETES_REQUEST_TIMEOUT	How long to wait for generic sequences of Kubernetes API calls.
LAB_RECONCILE_INTERVAL	How frequently to refresh user lab state from Kubernetes.
LIMIT_TO_REQUEST_RATIO	Ratio of lab resource limits to requests.
MEMORY_TO_TMP_SIZE_RATIO	Ratio of maximum memory size to maximum size of /tmp filesystem.
METADATA_PATH	Default path to injected pod metadata.
PREPULLER_POD_TIMEOUT	How long to wait for a prepuller pod to spawn and finish running.
RESERVED_ENV	Environment variables reserved to the controller or JupyterHub.
RESERVED_PATHS	Paths within the lab that are reserved for special purposes.
USERNAME_REGEX	Regex matching all valid usernames.

controller.dependencies.config Module

Config dependency.

Classes

ConfigDependency([path])

Dependency to manage a cached Nublado controller configuration.

Class Inheritance Diagram

controller.dependencies.context Module

Request context management.

ContextDependency is an all-in-one dependency, because managing individual dependencies turned out to be a real pain. It’s designed to capture the context of any request. It requires that a Config object has been loaded before it can be instantiated.

Classes

ContextDependency()	Provide a per-request context as a FastAPI dependency.
RequestContext(request, logger, factory, ...)	Holds the incoming request and its surrounding context.

Class Inheritance Diagram

controller.dependencies.user Module

User and authentication dependencies for FastAPI.

Functions

user_dependency(context, ...)	Return the validated user for the given request.
username_path_dependency(username, ...)	Validate and return the username in the request path.

controller.exceptions Module

Exceptions for the Nublado lab controller.

Classes

ControllerTimeoutError(operation[, user])	Wraps TimeoutError with additional context and Slack support.
DockerRegistryError(message, *[, failed_at, ...])	An API call to a Docker Registry failed.
DuplicateObjectError(message, *[, user, ...])	Multiple Kubernetes objects were found when one was expected.
GafaelfawrParseError(message, error)	Unable to parse the reply from Gafaelfawr.
GafaelfawrWebError(message, *[, failed_at, ...])	An API call to Gafaelfawr failed.
InsufficientQuotaError(message[, location, ...])	The user's quota is insufficient to satisfy this request.
InvalidDockerReferenceError(message[, ...])	Docker reference does not contain a tag.
InvalidLabSizeError(size)	The provided lab size is not one of the configured sizes.
InvalidTokenError(message[, location, ...])	The delegated user token is invalid.
KubernetesError(message, *[, user, kind, ...])	An API call to Kubernetes failed.
LabDeletionError(message[, user, failed_at])	An error occurred when deleting a lab.
LabExistsError(message[, location, field_path])	Lab creation was attempted for a user with an active lab.
MissingObjectError(message, *[, user, ...])	An expected Kubernetes object is missing.
MissingSecretError(name, namespace[, key])	Secret specified in the controller configuration was not found.
NoOperationError(message[, user, failed_at])	No operation was in progress when attempting to wait.
NotConfiguredError(message[, location, ...])	An attempt was made to use a disabled service.
OperationConflictError(username)	Attempt to perform an operation when another is in progress.
PermissionDeniedError(message[, location, ...])	Attempt to access a resource for another user.
UnknownDockerImageError(message[, location, ...])	Cannot find a Docker image matching the requested parameters.
UnknownUserError(message[, location, field_path])	No resource has been created for this user.

Class Inheritance Diagram

controller.factory Module

Component factory and global and per-request context management.

Classes

Factory(context, logger)	Build lab controller components.
ProcessContext(config, http_client, ...)	Per-process global application state.

Class Inheritance Diagram

controller.main Module

The main application factory for the Nublado controller service.

Functions

create_app(*[, load_config])

Create the FastAPI application.

controller.models.domain.docker Module

Domain models for talking to the Docker API.

Classes

DockerCredentials(username, password)	Holds the credentials for one Docker API server.
DockerReference(registry, repository, tag, ...)	Parses a Docker reference.

Class Inheritance Diagram

controller.models.domain.fileserver Module

Models for the fileserver state.

Classes

FileserverObjects(pvcs, ingress, service, job)	All of the Kubernetes objects making up a user's fileserver.
FileserverStateObjects(job, pod, ingress)	Kubernetes objects used for inspecting the state of a fileserver.

Class Inheritance Diagram

controller.models.domain.gafaelfawr Module

Models for talking to Gafaelfawr.

Ideally, these should use the same models Gafaelfawr itself uses. Until that’s possible via a PyPI library, these models are largely copied from Gafaelfawr.

Classes

GafaelfawrUser	User information from Gafaelfawr supplemented with the user's token.
GafaelfawrUserInfo	User metadata from Gafaelfawr.
NotebookQuota	Notebook Aspect quota information for a user.
UserGroup	Gafaelfawr's representation of a single group.
UserQuota	Quota information for a user.

Class Inheritance Diagram

controller.models.domain.image Module

Internal models returned by image service methods.

Classes

MenuImage(reference, name)	A single spawnable image.
MenuImages(menu, dropdown)	All available spawnable images.

Class Inheritance Diagram

controller.models.domain.kubernetes Module

Data types for interacting with Kubernetes.

Classes

Affinity	Pod affinity rules.
KubernetesModel(*args, **kwargs)	Protocol for Kubernetes object models.
KubernetesNodeImage(references, size)	A cached image on a Kubernetes node.
LabelSelector	Rule for matching labels of pods or namespaces.
LabelSelectorOperator(value[, names, ...])	Match operations for label selectors.
LabelSelectorRequirement	Single rule for label matching.
NodeAffinity	Node affinity rules.
NodeSelectorOperator(value[, names, module, ...])	Match operations for node selectors.
NodeSelectorRequirement	Individual match rule for nodes.
NodeSelectorTerm	Term to match nodes.
NodeSelector	Matching terms for nodes.
NodeToleration(eligible[, comment])	Whether a single node is tolerated.
PodAffinity	Pod affinity rules.
PodAffinityTerm	Pod matching term for pod affinity.
PodChange(phase, pod)	Represents a change (not creation or deletion) of a pod.
PodPhase(value[, names, module, qualname, ...])	One of the valid phases reported in the status section of a Pod.
PreferredSchedulingTerm	Scheduling term with a weight, used to find preferred nodes.
PropagationPolicy(value[, names, module, ...])	Possible values for the propagationPolicy parameter to delete.
PullPolicy(value[, names, module, qualname, ...])	Pull policy for Docker images in Kubernetes.
TaintEffect(value[, names, module, ...])	Possible effects of a pod toleration.
Toleration	Represents a single pod toleration rule.
TolerationOperator(value[, names, module, ...])	Possible operators for a toleration.
VolumeAccessMode(value[, names, module, ...])	Access mode for a persistent volume.
WeightedPodAffinityTerm	Pod matching term for pod affinity with an associated weight.
WatchEventType(value[, names, module, ...])	Possible values of the type field of Kubernetes watch events.

Class Inheritance Diagram

controller.models.domain.lab Module

Internal models related to user labs.

Classes

EventType(value[, names, module, qualname, ...])	Type of message.
Event	One lab operation event for a user.
LabObjectNames(username, namespace, ...)	Names for the key Kubernetes objects making up a user's lab.
LabObjects(env_config_map, quota, pod, ...)	All of the Kubernetes objects making up a user's lab.
LabStateObjects(env_config_map, quota, pod)	All of the Kubernetes objects required to reconstruct lab state.

Class Inheritance Diagram

controller.models.domain.rspimage Module

Abstract data types for handling RSP images.

Classes

RSPImage(tag, image_type, version, cycle, ...)	A tagged Rubin Science Platform image.
RSPImageCollection(images[, cycle])	Provides operations on a collection of RSPImage objects.

Class Inheritance Diagram

controller.models.domain.rsptag Module

Abstract data types for handling RSP image tags.

Classes

RSPImageTag(tag, image_type, version, cycle, ...)	A sortable image tag for a Rubin Science Platform image.
RSPImageTagCollection(tags)	Hold and perform operations on a set of RSPImageTag objects.
RSPImageType(value[, names, module, ...])	The type (generally, release series) of the identified image.

Variables

DOCKER_DEFAULT_TAG

Implicit tag used by Docker/Kubernetes when no tag is specified.

Class Inheritance Diagram

controller.models.domain.volumes Module

Models for mounted volumes.

Classes

MountedVolume(volume, volume_mount)

Represents a volume along with its corresponding mount.

Class Inheritance Diagram

controller.models.index Module

Top-level request models for the Nublado controller.

Classes

Index

Metadata returned by the external root URL of the application.

Class Inheritance Diagram

controller.models.v1.fileserver Module

API-visible models for user file servers.

Classes

FileserverStatus

Status of a user's file server.

Class Inheritance Diagram

controller.models.v1.lab Module

API-visible models for user lab environments.

Classes

CommonLabOptions	Options shared by both the input and output model for lab options.
ImageClass(value[, names, module, qualname, ...])	Supported classes of images.
LabOptions	Options for the lab, specified at creation time.
LabRequestOptions	User-provided lab configuration options.
LabResources	Resource requests and limits for a lab.
LabSize(value[, names, module, qualname, ...])	Allowable names for pod sizes.
LabSpecification	Specification of lab to spawn, sent by the JupyterHub spawner.
LabState	Current state of the user's lab.
LabStatus(value[, names, module, qualname, ...])	Possible states the user's lab may be in.
ResourceQuantity	A Kubernetes resource request or limit.
UserInfo	Metadata about the user who owns the lab.

Class Inheritance Diagram

controller.models.v1.prepuller Module

Models for prepuller.

Classes

DockerSourceOptions	Docker Registry from which to get images.
GARSourceOptions	Google Artifact Registry from which to get images.
Image	A single Docker image that is available or prepulled.
Node	Information about available images on a single Kubernetes node.
NodeImage	An available image present on at least some Kubernetes nodes.
PrepulledImage	Used to display available images.
PrepullerImageStatus	Status of the images being prepulled.
PrepullerOptions	Options for the prepuller.
PrepullerStatus	Status of the image prepuller.
SpawnerImages	Images known to the Nublado controller and available for spawning.

Class Inheritance Diagram

controller.services.builder.fileserver Module

Construction of Kubernetes objects for user fileservers.

Classes

FileserverBuilder(*, config, base_url, ...)

Construct Kubernetes objects for user file servers.

Class Inheritance Diagram

controller.services.builder.lab Module

Construction of Kubernetes objects for user lab environments.

Classes

LabBuilder(config, base_url, logger)

Construct Kubernetes objects for user lab environments.

Class Inheritance Diagram

controller.services.builder.prepuller Module

Construct Kubernetes objects used by the prepuller.

Classes

PrepullerBuilder(metadata_storage[, pull_secret])

Construct Kubernetes objects used by the prepuller.

Class Inheritance Diagram

controller.services.builder.volumes Module

Construction of Kubernetes objects for volumes and volume mounts.

Classes

VolumeBuilder()

Construct Kubernetes objects for volumes and volume mounts.

Class Inheritance Diagram

controller.services.fileserver Module

Service to manage user fileservers.

Classes

FileserverManager(*, config, ...)

Manage user file servers.

Class Inheritance Diagram

controller.services.image Module

Container image service.

Classes

ImageService(*, config, node_selector, ...)

Service to track the available images for Jupyter labs.

Class Inheritance Diagram

controller.services.lab Module

Service to manage user lab environments.

Classes

LabManager(*, config, image_service, ...)

Manage user labs.

Class Inheritance Diagram

controller.services.prepuller Module

Prepull images to Kubernetes nodes.

Classes

Prepuller(*, image_service, ...[, slack_client])

Prepull images to Kubernetes nodes.

Class Inheritance Diagram

controller.services.source.base Module

Base class for image sources.

Classes

ImageSource(logger)

Base class for image sources.

Class Inheritance Diagram

controller.services.source.docker Module

Image source using a Docker Registry.

Classes

DockerImageSource(config, docker, logger)

Image source using a Docker Registry.

Class Inheritance Diagram

controller.services.source.gar Module

Image source using a Google Artifact Registry.

Classes

GARImageSource(config, gar, logger)

Image source using a Google Artifact Registry.

Class Inheritance Diagram

controller.storage.docker Module

Client for the Docker v2 API.

Classes

DockerCredentialStore(credentials)	Read and write the .dockerconfigjson syntax used by Kubernetes.
DockerStorageClient(*, credentials_path, ...)	Client to query the Docker API for image information.

Class Inheritance Diagram

controller.storage.gafaelfawr Module

Client for talking to Gafaelfawr.

Classes

GafaelfawrStorageClient(config, http_client, ...)

Get user information from Gafaelfawr.

Class Inheritance Diagram

controller.storage.gar Module

Client for Google Artifact Registry.

Classes

GARStorageClient(logger)

Client for Google Artifact Registry.

Class Inheritance Diagram

controller.storage.kubernetes.creator Module

Generic Kubernetes object storage supporting only create and read.

Provides a generic Kubernetes object management class and instantiations of that class for Kubernetes object types that only support create and read. This is sufficient for a lot of object types the lab controller manipulates. Storage classes for object types that only need those operations are provided here.

For object types that need to support other operations, see KubernetesObjectDeleter, which subclasses KubernetesObjectCreator and adds list and delete support, and its subclasses.

Classes

ConfigMapStorage(api_client, logger)	Storage layer for ConfigMap objects.
KubernetesObjectCreator(*, create_method, ...)	Generic Kubernetes object storage supporting create and read.
NetworkPolicyStorage(api_client, logger)	Storage layer for NetworkPolicy objects.
ResourceQuotaStorage(api_client, logger)	Storage layer for ResourceQuota objects.
SecretStorage(api_client, logger)	Storage layer for Secret objects.

Variables

T	Type of Kubernetes object being manipulated.

Class Inheritance Diagram

controller.storage.kubernetes.custom Module

Storage layer for Kubernetes custom objects objects.

Classes

CustomStorage(*, api_client, group, version, ...)	Storage layer for Kubernetes custom objects.
GafaelfawrIngressStorage(api_client, logger)	Storage layer for GafaelfawrIngress objects.

Class Inheritance Diagram

controller.storage.kubernetes.deleter Module

Generic Kubernetes object storage including list and delete.

Provides a generic Kubernetes object management class and instantiations of that class for Kubernetes object types that support list and delete (as well as create and read, provided by the superclass). Storage classes for object types that only need those operations are provided here; more complex storage classes with other operations are defined in their own modules.

Classes

KubernetesObjectDeleter(*, create_method, ...)	Generic Kubernetes object storage supporting list and delete.
JobStorage(api_client, logger)	Storage layer for Job objects.
PersistentVolumeClaimStorage(api_client, logger)	Storage layer for PersistentVolumeClaim objects.
ServiceStorage(api_client, logger)	Storage layer for Service objects.

Variables

T	Type of Kubernetes object being manipulated.

Class Inheritance Diagram

controller.storage.kubernetes.fileserver Module

Kubernetes storage layer for user fileservers.

Classes

FileserverStorage(api_client, logger)

Kubernetes storage layer for file servers.

Class Inheritance Diagram

controller.storage.kubernetes.ingress Module

Storage layer for Ingress objects.

Functions

ingress_has_ip_address(ingress)

Check whether an ingress has an IP address assigned.

Classes

IngressStorage(api_client, logger)

Storage layer for Ingress objects.

Class Inheritance Diagram

controller.storage.kubernetes.lab Module

Kubernetes storage layer for user labs.

Classes

LabStorage(api_client, logger)

Kubernetes storage layer for user labs.

Class Inheritance Diagram

controller.storage.kubernetes.namespace Module

Storage layer for Namespace objects.

Classes

NamespaceStorage(api_client, logger)

Storage layer for Namespace objects.

Class Inheritance Diagram

controller.storage.kubernetes.node Module

Storage layer for Kubernetes node objects.

Classes

NodeStorage(api_client, logger)

Storage layer for Kubernetes node objects.

Class Inheritance Diagram

controller.storage.kubernetes.pod Module

Storage layer for Pod objects.

Classes

PodStorage(api_client, logger)

Storage layer for Pod objects.

Class Inheritance Diagram

controller.storage.kubernetes.watcher Module

Watch a Kubernetes namespace or cluster for events.

Classes

KubernetesWatcher(*, method, object_type, kind)	Watch Kubernetes for events.
WatchEvent(action, object)	Parsed event from a Kubernetes watch.

Variables

T	Type of Kubernetes object being watched (dict for custom objects).

Class Inheritance Diagram

controller.storage.metadata Module

Storage layer for pod metadata.

Classes

MetadataStorage(metadata_path)

Storage layer for pod metadata for the running pod.

Class Inheritance Diagram

controller.templates Module

Template management.

Provides a shared Jinja template environment used whenever the Nublado controller generates responses based on templates.

Variables

templates

The template manager.

controller.timeout Module

Timeout class for Kubernetes operations.

Classes

Timeout(operation, timeout[, user])

Track a cumulative timeout on a series of operations.

Class Inheritance Diagram

controller.units Module

Unit conversions for the Nublado controller.

Functions

memory_to_bytes(memory)

Convert a string representatio of memory to a number of bytes.