Reference

Locations

class tfnz.location.Location(*, location: Union[str, NoneType] = None, location_ip: Union[str, NoneType] = None, quiet: Union[bool, NoneType] = False, debug_log: Union[bool, NoneType] = False, new_node_callback: Optional = None)

The root location object.

Parameters:
  • location – An optional fqdn of the location (i.e. tiny.20ft.nz).
  • location_ip – A optional explicit ip for the broker.
  • quiet – Set true to not configure logging.
  • debug_log – Set true to log at DEBUG logging level.
  • new_node_callback – An optional callback for when a node is created … signature (object)
run()

Wait until the message loop completes, may raise an exception passed from the background thread.

complete(container=None, returncode=0)

Stop the background loop, causes ‘run’ to return. Call to close from the background thread.

disconnect(container=None, returncode=0)

Disconnect from the location - without calling this the object cannot be garbage collected

node() → tfnz.node.Node

Returns a node.

Returns:A node object
ranked_nodes() → List[tfnz.node.Node]

Ranks the nodes in order of resource availability.

Returns:A list of node objects.
create_volume(*, tag: Union[str, NoneType] = None, async: Union[bool, NoneType] = True, termination_callback: Optional = None) → tfnz.volume.Volume

Creates a new volume

Parameters:
  • tag – An optional globally visible tag.
  • async – Enables asynchronous writes.
  • termination_callback – a callback if this volume is destroyed - signature (container, returncode).
Returns:

The new Volume object.

Note that asynchronous writes cannot damage a ZFS filesystem although the physical state may lag behind the logical state by a number of seconds. Asynchronous ZFS is very much faster than synchronous.

ensure_volume(key: Union[bytes, str]) → tfnz.volume.Volume

Return the volume with this uuid, tag or display_name - create the volume if it doesn’t exist.

Parameters:key – The uuid or tag of the volume object to be returned.
Returns:A Volume object.
destroy_volume(volume: tfnz.volume.Volume)

Destroys an existing volume. This is not a ‘move to trash’, it will be destroyed.

Parameters:volume – The volume to be destroyed.
all_volumes() → List[tfnz.volume.Volume]

Returns a list of all volumes on this node.

Returns:A list of Volume objects.
volume(key: Union[bytes, str]) → tfnz.volume.Volume

Return the volume with this uuid, tag or display_name.

Parameters:key – The uuid or tag of the volume object to be returned.
Returns:A Volume object.
endpoint_for(fqdn: str) → tfnz.endpoint.WebEndpoint

Return a WebEndpoint for the given fqdn.

Parameters:fqdn – The fully qualified name the endpoint will represent.
Returns:A WebEndpoint object.
external_container(key: Union[bytes, str]) → tfnz.container.ExternalContainer

Return the external container with this uuid, tag or display_name.

Parameters:key – The uuid or tag of the container to be returned.
Returns:An ExternalContainer object.
ensure_image_uploaded(docker_image_id: str, *, descr: Union[dict, NoneType] = None) → List[str]

Sends missing docker layers to the location.

Parameters:
  • docker_image_id – use the short form id or tag
  • descr – a previously found docker description
Returns:

A list of layer sha256 identifiers

This is not a necessary step and is implied when spawning a container.

static all_locations()

Returns a (text) list of 20ft locations that have an account on this machine.

Nodes

class tfnz.node.Node(parent, pk, conn, stats)

An object representing a single node. Do not construct directly, use Location.node() or Location.ranked_nodes().

spawn_container(image: str, *, env: Union[typing.List[typing.Tuple[str, str]], NoneType] = None, sleep: Union[bool, NoneType] = False, volumes: Union[typing.List[typing.Tuple[tfnz.volume.Volume, str]], NoneType] = None, pre_boot_files: Union[typing.List[typing.Tuple[str, bytes]], NoneType] = None, command: Union[str, NoneType] = None, stdout_callback: Optional = None, termination_callback: Optional = None, tag: Union[str, NoneType] = None) → tfnz.container.Container

Asynchronously spawns a container on the node.

Parameters:
  • image – either the tag or short image id from Docker.
  • env – a list of environment name, value pairs to be passed.
  • sleep – replaces the Entrypoint/Cmd with a single blocking command (container still boots).
  • volumes – a list of (volume, mountpoint) pairs.
  • pre_boot_files – a list of (filename, data) pairs to write into the container before booting.
  • command – ignores Entrypoint/Cmd and launches with this script instead, list not string.
  • stdout_callback – Called when the container sends output - signature (container, string).
  • termination_callback – For when the container completes - signature (container, returncode).
  • tag – A tag used so other sessions (for the same user) can reference the container.
Returns:

A Container object.

The resulting Container is initially a placeholder until the container has spawned. Any layers that need to be uploaded to the location are uploaded automatically. Note that the container will not be marked as ready until it actually has booted.

To launch synchronously call wait_until_ready() on the container.

destroy_container(container: tfnz.container.Container)

Destroy a container running on this node. Will also destroy any tunnels onto the container.

Parameters:container – The container to be destroyed.
all_containers() → List[tfnz.container.Container]

Returns a list of all the containers running on this node (for this session). Does not include external containers.

Returns:A list of Container objects.
stats() → dict

Returns a dictionary describing this nodes’ current performance.

Volumes

class tfnz.volume.Volume(location, uuid, tag, *, termination_callback=None)

An object representing a persistent volume. Do not construct directly, use Location.create_volume or Location.ensure_volume; or Location.volume to retrieve one that is pre-existing.

snapshot()

Create a snapshot.

rollback()

Resets the volume back to its’ state when ‘snapshot’ was called.

Containers

class tfnz.container.Container(parent, image, uuid, docker_config, env, volumes, *, stdout_callback, termination_callback)

An object representing a single container. Do not instantiate directly, use node.spawn.

start()

Start a container that was spawned with sleep=True

private_ip()

Reports the container’s ip address

node()

Returns the node object this container is running on

stdin(data: bytes)

Writes the data into the container’s stdin.

Parameters:data – The data to be written.
wait_tcp(dest_port)

Connects and disconnects a single connection onto this container and a given port.

If your service is only capable of .accept’ing once, do not use this.

Parameters:dest_port – destination tcp port < 1024 is fine.
wait_http_200(*, dest_port: Union[int, NoneType] = 80, fqdn: Union[str, NoneType] = 'localhost', path: Union[str, NoneType] = '') → tfnz.tunnel.Tunnel

Poll until an http 200 is returned.

Parameters:
  • dest_port – Override the default port.
  • fqdn – A host name to use in the http request.
  • path – A path on the server - appended to /
Returns:

A Tunnel object.

attach_tunnel(dest_port: int, *, localport: Union[int, NoneType] = None, bind: Union[str, NoneType] = None) → tfnz.tunnel.Tunnel

Creates a TCP proxy between localhost and a container.

Parameters:
  • dest_port – The TCP port on the container to connect to.
  • localport – Optional choice of local port no - if not provided uses the dest_port.
  • bind – Optionally bind to an address other than localhost.
Returns:

A Tunnel object.

This call does no checking to ensure the server side is ready - but a failed connection will not destroy the tunnel itself and will poll until connected.

destroy_tunnel(tunnel: tfnz.tunnel.Tunnel)

Destroy a tunnel

Parameters:tunnel – The tunnel to be destroyed.
all_tunnels() → List[tfnz.tunnel.Tunnel]

Returns all the tunnels connected to this container

Returns:A list of Tunnel objects
allow_connection_from(container: tfnz.container.Container)

Allow another container to call this one over private ip

Parameters:container – The container that will be allowed to call.
disallow_connection_from(container: tfnz.container.Container)

Stop allowing another container to call this one over private ip

Parameters:container – The container that will no longer be allowed to call.
spawn_process(remote_command: str, data_callback: Union[typing.Callable, NoneType] = None, termination_callback: Union[typing.Callable, NoneType] = None, stderr_callback: Union[typing.Callable, NoneType] = None) → tfnz.process.Process

Spawn a process within a container, receives data asynchronously via a callback.

Parameters:
  • remote_command – The command to remotely launch as a string (i.e. not list).
  • data_callback – A callback for arriving data - signature (object, bytes).
  • termination_callback – For when the process completes - signature (object), returncode.
  • stderr_callback – For data emitted by the process’s stderr stream - signature (object, bytes).
Returns:

A Process object.

run_process(remote_command: str, *, data_callback: Union[typing.Callable, NoneType] = None, stderr_callback: Union[typing.Callable, NoneType] = None, nolog: Union[bool, NoneType] = False) -> (<class 'bytes'>, <class 'bytes'>, <class 'str'>)

Run a process once, synchronously, without pty.

Parameters:
  • remote_command – The command to run remotely.
  • data_callback – A callback for arriving data - signature (object, bytes).
  • stderr_callback – For data emitted by the process’s stderr stream - signature (object, bytes).
  • nolog – Don’t log this command (to hide sensitive data).
Returns:

stdout from the process, stderr from the process, exit code (as string).

spawn_shell(*, data_callback: Union[typing.Callable, NoneType] = None, termination_callback: Union[typing.Callable, NoneType] = None, echo: bool = False) → tfnz.process.Process

Spawn a shell within a container, expose as a process

Parameters:
  • data_callback – A callback for arriving data - signature (object, bytes).
  • termination_callback – For when the process completes - signature (object, returncode).
  • echo – Whether or not the shell echoes input.
Returns:

A Process object.

destroy_process(process: tfnz.process.Process)

Destroy a process or shell

Parameters:process – The process to be destroyed.
all_processes() → List[tfnz.process.Process]

Returns all the processes (launched via API) running on this container.

Returns:A list of Process objects
create_ssh_server(port: int = 2222) → tfnz.ssh.SshServer

Create an ssh/sftp server on the given port.

Parameters:port – Local tcp port number
Returns:An SshServer object.
destroy_ssh_server(server)

Destroy an ssh/sftp server attached to this container.

Parameters:server – An SshServer object.
fetch(filename: str) → bytes

Fetch a single file from the container.

Parameters:filename – The full-path name of the file to be retrieved.
Returns:the contents of the file as a bytes object.

Since the file gets loaded into memory, this is a bad way to move large files (>1GB).

put(filename: str, data: bytes)

Put a file into the container.

Parameters:
  • filename – The full-path name of the file to be placed.
  • data – The contents of the file as a bytes object.

This will just overwrite so be careful. Note that new file paths are created on demand. Similarly to fetch, this is a bad way to move large files (>1GB).

reboot(*, reset_filesystem: Union[bool, NoneType] = False)

Synchronously reboot a container, optionally resetting the filesystem.

Parameters:reset_filesystem – Reset the container’s filesystem to its ‘as booted’ state.
class tfnz.container.ExternalContainer(loc, uuid, node, ip, tag, *, termination_callback: Optional = None)

An object representing a container managed by another session (and the same user) but advertised using a tag. Do not instantiate directly, use Location.external_container

private_ip()

Reports the container’s ip address

Processes

class tfnz.process.Process(parent, uuid, data_callback, termination_callback, stderr_callback=None)

An object encapsulating a process within a container. Do not instantiate directly, use container.spawn_process.

stdin(data: bytes)

Inject data into stdin for the process.

Parameters:data – The data to inject - bytes, not a string.

Note that because this injects raw data, it may not behave as you expect. Remember to:

  • Turn strings into bytes with .encode()
  • Add ‘\n’ to emulate carriage return.
  • Turn returned bytes into strings with .decode()

Tunnels

class tfnz.tunnel.Tunnel(connection, node, container, port, lp=None, bind=None, timeout=30)

An object representing a TCP proxy from localhost onto a container. Do not instantiate directly, use location.tunnel_onto or location.wait_http_200.

Interact with the proxy through TCP (or call localport if you didn’t set it explicitly). Note that apparently plaintext traffic through the tunnel is still encrypted on the wire.

localport() → int

Returns the (possibly dynamically allocated) local port number.

Returns:the port number as an int

Endpoints

class tfnz.endpoint.Cluster(*, containers: Union[typing.List[tfnz.container.Container], NoneType] = None, rewrite: Union[str, NoneType] = None)

An object representing a collection of containers, load balanced and published to an endpoint.

Parameters:
  • containers – An optional list of containers to initialise the cluster with.
  • rewrite – An optional string to be rewritten into the http host header.
add_container(container)

Add a container to the cluster.

Parameters:container – the container to add.
remove_container(container)

Remove a container from the cluster.

Parameters:container – the container to remove.
class tfnz.endpoint.WebEndpoint(location, domain: str)

An HTTP proxy that can expose a number of clusters onto a domain.

publish(cluster: tfnz.endpoint.Cluster, fqdn: str, *, ssl: Union[typing.Tuple, NoneType] = None)

Publish a cluster onto an http/https endpoint. To update a cluster, merely re-publish onto the same endpoint.

Parameters:
  • cluster – The cluster to publish.
  • fqdn – The fqdn to publish.
  • ssl – A tuple of (cert.pem, key.pem) or (cert.pem, key.pem, cert.intermediate).
static wait_http_200(*, ssl: Union[bool, NoneType] = False)

Poll the gateway for an http 200 from this cluster.

Parameters:
  • fqdn – the fqdn to poll.
  • ssl – optionally connect via ssl.
unpublish(cluster: tfnz.endpoint.Cluster)

Remove a cluster from a web endpoint.

Parameters:cluster – the cluster to remove.

Traits

class tfnz.Waitable(locked: Union[bool, NoneType] = True)

An object that can be waited on (until marked as ready)

wait_until_ready(timeout: Union[int, NoneType] = 60)

Blocks waiting for a (normally asynchronous) update indicating the object is ready.

Parameters:timeout – An optional timeout in seconds.
Returns:self
is_ready() → bool
Returns:True if object is ready.
class tfnz.Connectable(conn: messidge.client.connection.Connection, uuid: str, node, ip)

A resource that can be connected to in the private IP space.

allow_connection_from(obj)

Allow bidirectional communication between this object and the parameter.

Parameters:obj – An object that has an ip.
disallow_connection_from(obj)

Disallow bidirectional communication between this object and one that was previously allowed to connect.

Parameters:obj – An object that has an ip.
class tfnz.Taggable(user: Union[str, bytes], uuid: Union[str, bytes], tag: Union[str, bytes, NoneType] = None)

A resource that might have a globally advertisable tag.

uuid_key() -> (<class 'bytes'>, <class 'bytes'>)
Returns:the owner (user) pk and uuid of this object.
tag_key() -> (<class 'bytes'>, <class 'bytes'>)
Returns:the owner (user) pk and tag of this object (or None).
display_name() → str
Returns:the object’s uuid, or uuid:tag pair (as a string) if possible.
class tfnz.TaggedCollection(initial: Union[typing.List[tfnz.Taggable], NoneType] = None)

A collection of taggable objects.

__call__(*args, **kwargs) → set

Calling the tagged collection as if a function.

Returns:a set of unique values
__len__() → int

Calling len on the collection.

Returns:the number of unique values.
__getitem__(uuid: Union[str, bytes])

Dereference the collection just as you would for a dict, passing the uuid.

Parameters:uuid – the uuid of the object to retrieve.
Returns:the object (or raises KeyError).
__contains__(uuid: Union[str, bytes]) → bool

Test to see if a given uuid is ‘in’ the collection.

Parameters:uuid – the uuid of the object to test for.
Returns:boolean
get(user: Union[bytes, str], key: Union[bytes, str])

Fetch an object given the user PK and a uuid, tag, or uuid:tag

Parameters:
  • user – User PK
  • key – Either the object uuid, tag, or ‘uuid:tag’ as a format.
Returns:

The reference object or a KeyError.

values() → set
Returns:A set of unique values

Components

class tfnz.components.postgresql.Postgresql(node: tfnz.node.Node, volume: tfnz.volume.Volume, *, password: str = None, log_callback=None, image: str = None)

An object encapsulating a Postgresql server running on it’s default port (5432). Connect with username=postgres.

Parameters:
  • node – The node to spawn on.
  • volume – A volume (object) to use as a persistent store.
  • password – An optional password for the database, will create one if not supplied.
  • log_callback – An optional callback for log messages - signature (object, bytes)
  • image – Specify a non-default image.

Note the instantiated object behaves as if it were derived from Container.

password() → str
Returns:password for the server.
wait_truly_up()

Wait on the server until it is ready to answer queries.

ensure_database(name: str) → bool

Ensures a given database exists in this server. Returns True if it had to be created.

Platforms

class tfnz.platforms.mezzanine.Mezzanine(location: tfnz.location.Location, volume: tfnz.volume.Volume, sql_volume: tfnz.volume.Volume, fqdn: str, app_name: str, image, *, log_callback=None, superuser=None, debug=False)
container_id = 'cc06f4404bda'

Puts a Python/Django/Mezzanine instance on each node and load balances.

Parameters:
  • location – A location (object) to connect to.
  • volume – A volume (object) to use as a persistent store - will be mounted on static/media/uploads
  • sql_volume – A volume to connect to a Postgres server for SQL storage.
  • fqdn – The FQDN to publish to.
  • app_name – Name of the application (used to create file paths).
  • image – The image to use (assumed derived ‘FROM tfnz/mezzanine’)
  • log_callback – An optional callback for log messages - signature (object, bytes)
  • superuser – if creating from fresh set this (username, email) as the admin user
  • debug – Whether or not to set debug on django.
change_password(username, password)

change the username and password of the given user

class tfnz.platforms.silverstripe.SilverStripe(location: tfnz.location.Location, volume: tfnz.volume.Volume, sql_volume: tfnz.volume.Volume, fqdn: str, *, image=None, log_callback=None)
container_id = '7a4f9fbb6afc'

Puts a PHP/SilverStripe instance on each node and load balances.

Parameters:
  • location – A location (object) to connect to.
  • volume – A volume (object) to use as a persistent store.
  • sql_volume – A volume to connect to a Postgres server for SQL storage.
  • fqdn – The FQDN to publish to.
  • image – Use a non-default container image.
  • log_callback – An optional callback for log messages - signature (object, bytes)

CLIs

tfnz.cli.base_argparse(progname, location=True) → argparse.ArgumentParser

Create an argparser with –location and –local flags.

Returns:An argparser
tfnz.cli.generic_cli(parser: argparse.ArgumentParser, implementations, *, quiet=True, location=True)

Call to implement a cli. See tfdomains etc.

Parameters:
  • parser – an ArgumentParser from base_argparse.
  • implementations – A map of verb->implementation for the various commands.
  • quiet – Don’t configure logging.
  • location – Whether or not to connect to the location.
class tfnz.cli.Interactive(container)

Wrap around a container to map stdin and stdout to terminal.

Parameters:container – the Container to wrap.
static stdout_callback(out)

Pass Interactive.stdout_callback as the stdout_callback parameter in spawn_container.

stop(obj=None, code=None)

Call to stop the background loop, can be a termination_callback parameter.

Misc

class tfnz.docker.Docker
static description(*, conn: Union[_ForwardRef('Connection'), NoneType] = None) → dict

Describe a docker image.

Parameters:
  • docker_image_id – Docker image id.
  • conn – An optional connection to the location.
Returns:

A dict representation of image metadata.

static tarball() → bytes

Retrieve the tarball of a docker image.

Parameters:docker_image_id – Docker image id.
Returns:A stream of bytes that would be the contents of the tar archive.
static last_image() → str

Finding the most recent docker image on this machine.

Returns:Docker image id of the most recently built docker image