Specification Version: 1.0.1

Open Container Initiative Runtime Specification

The Open Container Initiative develops specifications for standards on Operating System process and application containers.

Abstract

The Open Container Initiative Runtime Specification aims to specify the configuration, execution environment, and lifecycle of a container.

A container's configuration is specified as the config.json for the supported platforms and details the fields that enable the creation of a container.
The execution environment is specified to ensure that applications running inside a container have a consistent environment between runtimes along with common actions defined for the container's lifecycle.

Platforms

Platforms defined by this specification are:

Table of Contents

Notational Conventions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" are to be interpreted as described in RFC 2119.

The key words "unspecified", "undefined", and "implementation-defined" are to be interpreted as described in the rationale for the C99 standard.

An implementation is not compliant for a given CPU architecture if it fails to satisfy one or more of the MUST, REQUIRED, or SHALL requirements for the platforms it implements.
An implementation is compliant for a given CPU architecture if it satisfies all the MUST, REQUIRED, and SHALL requirements for the platforms it implements.

The 5 principles of Standard Containers

Define a unit of software delivery called a Standard Container.
The goal of a Standard Container is to encapsulate a software component and all its dependencies in a format that is self-describing and portable, so that any compliant runtime can run it without extra dependencies, regardless of the underlying machine and the contents of the container.

The specification for Standard Containers defines:

  1. configuration file formats
  2. a set of standard operations
  3. an execution environment.

A great analogy for this is the physical shipping container used by the transportation industry.
Shipping containers are a fundamental unit of delivery, they can be lifted, stacked, locked, loaded, unloaded and labelled.
Irrespective of their contents, by standardizing the container itself it allowed for a consistent, more streamlined and efficient set of processes to be defined.
For software Standard Containers offer similar functionality by being the fundamental, standardized, unit of delivery for a software package.

1. Standard operations

Standard Containers define a set of STANDARD OPERATIONS.
They can be created, started, and stopped using standard container tools; copied and snapshotted using standard filesystem tools; and downloaded and uploaded using standard network tools.

2. Content-agnostic

Standard Containers are CONTENT-AGNOSTIC: all standard operations have the same effect regardless of the contents.
They are started in the same way whether they contain a postgres database, a php application with its dependencies and application server, or Java build artifacts.

3. Infrastructure-agnostic

Standard Containers are INFRASTRUCTURE-AGNOSTIC: they can be run in any OCI supported infrastructure.
For example, a standard container can be bundled on a laptop, uploaded to cloud storage, downloaded, run and snapshotted by a build server at a fiber hotel in Virginia, uploaded to 10 staging servers in a home-made private cloud cluster, then sent to 30 production instances across 3 public cloud regions.

4. Designed for automation

Standard Containers are DESIGNED FOR AUTOMATION: because they offer the same standard operations regardless of content and infrastructure, Standard Containers, are extremely well-suited for automation.
In fact, you could say automation is their secret weapon.

Many things that once required time-consuming and error-prone human effort can now be programmed.
Before Standard Containers, by the time a software component ran in production, it had been individually built, configured, bundled, documented, patched, vendored, templated, tweaked and instrumented by 10 different people on 10 different computers.
Builds failed, libraries conflicted, mirrors crashed, post-it notes were lost, logs were misplaced, cluster updates were half-broken.
The process was slow, inefficient and cost a fortune - and was entirely different depending on the language and infrastructure provider.

5. Industrial-grade delivery

Standard Containers make INDUSTRIAL-GRADE DELIVERY of software a reality.
Leveraging all of the properties listed above, Standard Containers are enabling large and small enterprises to streamline and automate their software delivery pipelines.
Whether it is in-house devOps flows, or external customer-based software delivery mechanisms, Standard Containers are changing the way the community thinks about software packaging and delivery.

Filesystem Bundle

Container Format

This section defines a format for encoding a container as a filesystem bundle - a set of files organized in a certain way, and containing all the necessary data and metadata for any compliant runtime to perform all standard operations against it.
See also MacOS application bundles for a similar use of the term bundle.

The definition of a bundle is only concerned with how a container, and its configuration data, are stored on a local filesystem so that it can be consumed by a compliant runtime.

A Standard Container bundle contains all the information needed to load and run a container.
This includes the following artifacts:

  1. config.json: contains configuration data.
    This REQUIRED file MUST reside in the root of the bundle directory and MUST be named config.json.
    See     config.json for more details.

  2. container's root filesystem: the directory referenced by root.path, if that property is set in config.json.

When supplied, while these artifacts MUST all be present in a single directory on the local filesystem, that directory itself is not part of the bundle.
In other words, a tar archive of a bundle will have these artifacts at the root of the archive, not nested within a top-level directory.

Runtime and Lifecycle

Scope of a Container

The entity using a runtime to create a container MUST be able to use the operations defined in this specification against that same container.
Whether other entities using the same, or other, instance of the runtime can see that container is out of scope of this specification.

State

The state of a container includes the following properties:

The state MAY include additional properties.

When serialized in JSON, the format MUST adhere to the following pattern:

{
    "ociVersion": "0.2.0",
    "id": "oci-container1",
    "status": "running",
    "pid": 4422,
    "bundle": "/containers/redis",
    "annotations": {
        "myKey": "myValue"
    }
}

See Query State for information on retrieving the state of a container.

Lifecycle

The lifecycle describes the timeline of events that happen from when a container is created to when it ceases to exist.

  1. OCI compliant runtime's create command is invoked with a reference to the location of the bundle and a unique identifier.
  2. The container's runtime environment MUST be created according to the configuration in config.json.
    If the runtime is unable to create the environment specified in the config.json, it MUST generate an error.
    While the resources requested in the config.json MUST be created, the user-specified program (from process) MUST NOT be run at this time.
    Any updates to config.json after this step MUST NOT affect the container.
  3. Runtime's start command is invoked with the unique identifier of the container.
  4. The prestart hooks MUST be invoked by the runtime.
    If any prestart hook fails, the runtime MUST generate an error, stop the container, and continue the lifecycle at step 9.
  5. The runtime MUST run the user-specified program, as specified by process.
  6. The poststart hooks MUST be invoked by the runtime.
    If any poststart hook fails, the runtime MUST log a warning, but the remaining hooks and lifecycle continue as if the hook had succeeded.
  7. The container process exits.
    This MAY happen due to erroring out, exiting, crashing or the runtime's kill operation being invoked.
  8. Runtime's delete command is invoked with the unique identifier of the container.
  9. The container MUST be destroyed by undoing the steps performed during create phase (step 2).
  10. The poststop hooks MUST be invoked by the runtime.
    If any poststop hook fails, the runtime MUST log a warning, but the remaining hooks and lifecycle continue as if the hook had succeeded.

Errors

In cases where the specified operation generates an error, this specification does not mandate how, or even if, that error is returned or exposed to the user of an implementation.
Unless otherwise stated, generating an error MUST leave the state of the environment as if the operation were never attempted - modulo any possible trivial ancillary changes such as logging.

Warnings

In cases where the specified operation logs a warning, this specification does not mandate how, or even if, that warning is returned or exposed to the user of an implementation.
Unless otherwise stated, logging a warning does not change the flow of the operation; it MUST continue as if the warning had not been logged.

Operations

Unless otherwise stated, runtimes MUST support the following operations.

Note: these operations are not specifying any command-line APIs, and the parameters are inputs for general operations.

Query State

state <container-id>

This operation MUST generate an error if it is not provided the ID of a container.
Attempting to query a container that does not exist MUST generate an error.
This operation MUST return the state of a container as specified in the State section.

Create

create <container-id> <path-to-bundle>

This operation MUST generate an error if it is not provided a path to the bundle and the container ID to associate with the container.
If the ID provided is not unique across all containers within the scope of the runtime, or is not valid in any other way, the implementation MUST generate an error and a new container MUST NOT be created.
This operation MUST create a new container.

All of the properties configured in config.json except for process MUST be applied.
process.args MUST NOT be applied until triggered by the start operation.
The remaining process properties MAY be applied by this operation.
If the runtime cannot apply a property as specified in the configuration, it MUST generate an error and a new container MUST NOT be created.

The runtime MAY validate config.json against this spec, either generically or with respect to the local system capabilities, before creating the container (step 2).
Runtime callers who are interested in pre-create validation can run bundle-validation tools before invoking the create operation.

Any changes made to the config.json file after this operation will not have an effect on the container.

Start

start <container-id>

This operation MUST generate an error if it is not provided the container ID.
Attempting to start a container that is not created MUST have no effect on the container and MUST generate an error.
This operation MUST run the user-specified program as specified by process.
This operation MUST generate an error if process was not set.

Kill

kill <container-id> <signal>

This operation MUST generate an error if it is not provided the container ID.
Attempting to send a signal to a container that is neither created nor running MUST have no effect on the container and MUST generate an error.
This operation MUST send the specified signal to the container process.

Delete

delete <container-id>

This operation MUST generate an error if it is not provided the container ID.
Attempting to delete a container that is not stopped MUST have no effect on the container and MUST generate an error.
Deleting a container MUST delete the resources that were created during the create step.
Note that resources associated with the container, but not created by this container, MUST NOT be deleted.
Once a container is deleted its ID MAY be used by a subsequent container.

Hooks

Many of the operations specified in this specification have "hooks" that allow for additional actions to be taken before or after each operation.
See runtime configuration for hooks for more information.

Linux Runtime

File descriptors

By default, only the stdin, stdout and stderr file descriptors are kept open for the application by the runtime.
The runtime MAY pass additional file descriptors to the application to support features such as socket activation.
Some of the file descriptors MAY be redirected to /dev/null even though they are open.

While creating the container (step 2 in the lifecycle), runtimes MUST create the following symlinks if the source file exists after processing mounts:

Source Destination
/proc/self/fd /dev/fd
/proc/self/fd/0 /dev/stdin
/proc/self/fd/1 /dev/stdout
/proc/self/fd/2 /dev/stderr

Configuration

This configuration file contains metadata necessary to implement standard operations against the container.
This includes the process to run, environment variables to inject, sandboxing features to use, etc.

The canonical schema is defined in this document, but there is a JSON Schema in schema/config-schema.json and Go bindings in specs-go/config.go.
Platform-specific configuration schema are defined in the platform-specific documents linked below.
For properties that are only defined for some platforms, the Go property has a platform tag listing those protocols (e.g. platform:"linux,solaris").

Below is a detailed description of each field defined in the configuration format and valid values are specified.
Platform-specific fields are identified as such.
For all platform-specific configuration values, the scope defined below in the Platform-specific configuration section applies.

Specification version

Example

    "ociVersion": "0.1.0"

Root

root (object, OPTIONAL) specifies the container's root filesystem.
On Windows, for Windows Server Containers, this field is REQUIRED.
For Hyper-V Containers, this field MUST NOT be set.

On all other platforms, this field is REQUIRED.

Example (POSIX platforms)

"root": {
    "path": "rootfs",
    "readonly": true
}

Example (Windows)

"root": {
    "path": "\\\\?\\Volume{ec84d99e-3f02-11e7-ac6c-00155d7682cf}\\"
}

Mounts

mounts (array of objects, OPTIONAL) specifies additional mounts beyond root.
The runtime MUST mount entries in the listed order.
For Linux, the parameters are as documented in mount(2) system call man page.
For Solaris, the mount entry corresponds to the 'fs' resource in the zonecfg(1M) man page.

Example (Windows)

"mounts": [
    {
        "destination": "C:\\folder-inside-container",
        "source": "C:\\folder-on-host",
        "options": ["ro"]
    }
]

POSIX-platform Mounts

For POSIX platforms the mounts structure has the following fields:

Example (Linux)

"mounts": [
    {
        "destination": "/tmp",
        "type": "tmpfs",
        "source": "tmpfs",
        "options": ["nosuid","strictatime","mode=755","size=65536k"]
    },
    {
        "destination": "/data",
        "type": "bind",
        "source": "/volumes/testing",
        "options": ["rbind","rw"]
    }
]

Example (Solaris)

"mounts": [
    {
        "destination": "/opt/local",
        "type": "lofs",
        "source": "/usr/local",
        "options": ["ro","nodevices"]
    },
    {
        "destination": "/opt/sfw",
        "type": "lofs",
        "source": "/opt/sfw"
    }
]

Process

process (object, OPTIONAL) specifies the container process.
This property is REQUIRED when start is called.

POSIX process

For systems that support POSIX rlimits (for example Linux and Solaris), the process object supports the following process-specific properties:

Linux Process

For Linux-based systems, the process object supports the following process-specific properties.

User

The user for the process is a platform-specific structure that allows specific control over which user the process runs as.

POSIX-platform User

For POSIX platforms the user structure has the following fields:

Note: symbolic name for uid and gid, such as uname and gname respectively, are left to upper levels to derive (i.e. /etc/passwd parsing, NSS, etc)

Example (Linux)

"process": {
    "terminal": true,
    "consoleSize": {
        "height": 25,
        "width": 80
    },
    "user": {
        "uid": 1,
        "gid": 1,
        "additionalGids": [5, 6]
    },
    "env": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
        "TERM=xterm"
    ],
    "cwd": "/root",
    "args": [
        "sh"
    ],
    "apparmorProfile": "acme_secure_profile",
    "selinuxLabel": "system_u:system_r:svirt_lxc_net_t:s0:c124,c675",
    "noNewPrivileges": true,
    "capabilities": {
        "bounding": [
            "CAP_AUDIT_WRITE",
            "CAP_KILL",
            "CAP_NET_BIND_SERVICE"
        ],
       "permitted": [
            "CAP_AUDIT_WRITE",
            "CAP_KILL",
            "CAP_NET_BIND_SERVICE"
        ],
       "inheritable": [
            "CAP_AUDIT_WRITE",
            "CAP_KILL",
            "CAP_NET_BIND_SERVICE"
        ],
        "effective": [
            "CAP_AUDIT_WRITE",
            "CAP_KILL"
        ],
        "ambient": [
            "CAP_NET_BIND_SERVICE"
        ]
    },
    "rlimits": [
        {
            "type": "RLIMIT_NOFILE",
            "hard": 1024,
            "soft": 1024
        }
    ]
}

Example (Solaris)

"process": {
    "terminal": true,
    "consoleSize": {
        "height": 25,
        "width": 80
    },
    "user": {
        "uid": 1,
        "gid": 1,
        "additionalGids": [2, 8]
    },
    "env": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
        "TERM=xterm"
    ],
    "cwd": "/root",
    "args": [
        "/usr/bin/bash"
    ]
}

Windows User

For Windows based systems the user structure has the following fields:

Example (Windows)

"process": {
    "terminal": true,
    "user": {
        "username": "containeradministrator"
    },
    "env": [
        "VARIABLE=1"
    ],
    "cwd": "c:\\foo",
    "args": [
        "someapp.exe",
    ]
}

Hostname

Example

"hostname": "mrsdalloway"

Platform-specific configuration

Example (Linux)

{
    "linux": {
        "namespaces": [
            {
                "type": "pid"
            }
        ]
    }
}

POSIX-platform Hooks

For POSIX platforms, the configuration structure supports hooks for configuring custom actions related to the lifecycle of the container.

Hooks allow users to specify programs to run before or after various lifecycle events.
Hooks MUST be called in the listed order.
The state of the container MUST be passed to hooks over stdin so that they may do work appropriate to the current state of the container.

Prestart

The pre-start hooks MUST be called after the start operation is called but before the user-specified program command is executed.
On Linux, for example, they are called after the container namespaces are created, so they provide an opportunity to customize the container (e.g. the network namespace could be specified in this hook).

Poststart

The post-start hooks MUST be called after the user-specified process is executed but before the start operation returns.
For example, this hook can notify the user that the container process is spawned.

Poststop

The post-stop hooks MUST be called after the container is deleted but before the delete operation returns.
Cleanup or debugging functions are examples of such a hook.

Example

    "hooks": {
        "prestart": [
            {
                "path": "/usr/bin/fix-mounts",
                "args": ["fix-mounts", "arg1", "arg2"],
                "env":  [ "key1=value1"]
            },
            {
                "path": "/usr/bin/setup-network"
            }
        ],
        "poststart": [
            {
                "path": "/usr/bin/notify-start",
                "timeout": 5
            }
        ],
        "poststop": [
            {
                "path": "/usr/sbin/cleanup.sh",
                "args": ["cleanup.sh", "-f"]
            }
        ]
    }

Annotations

annotations (object, OPTIONAL) contains arbitrary metadata for the container.
This information MAY be structured or unstructured.
Annotations MUST be a key-value map.
If there are no annotations then this property MAY either be absent or an empty map.

Keys MUST be strings.
Keys MUST NOT be an empty string.
Keys SHOULD be named using a reverse domain notation - e.g. `com.example.myKey`.
Keys using the `org.opencontainers` namespace are reserved and MUST NOT be used by subsequent specifications.
Implementations that are reading/processing this configuration file MUST NOT generate an error if they encounter an unknown annotation key.

Values MUST be strings.
Values MAY be an empty string.
"annotations": {
    "com.example.gpu-cores": "2"
}

Extensibility

Runtimes that are reading or processing this configuration file MUST NOT generate an error if they encounter an unknown property.
Instead they MUST ignore unknown properties.

Valid values

Runtimes that are reading or processing this configuration file MUST generate an error when invalid or unsupported values are encountered.
Unless support for a valid value is explicitly required, runtimes MAY choose which subset of the valid values it will support.

Configuration Schema Example

Here is a full example config.json for reference.

{
    "ociVersion": "0.5.0-dev",
    "process": {
        "terminal": true,
        "user": {
            "uid": 1,
            "gid": 1,
            "additionalGids": [
                5,
                6
            ]
        },
        "args": [
            "sh"
        ],
        "env": [
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "TERM=xterm"
        ],
        "cwd": "/",
        "capabilities": {
            "bounding": [
                "CAP_AUDIT_WRITE",
                "CAP_KILL",
                "CAP_NET_BIND_SERVICE"
            ],
            "permitted": [
                "CAP_AUDIT_WRITE",
                "CAP_KILL",
                "CAP_NET_BIND_SERVICE"
            ],
            "inheritable": [
                "CAP_AUDIT_WRITE",
                "CAP_KILL",
                "CAP_NET_BIND_SERVICE"
            ],
            "effective": [
                "CAP_AUDIT_WRITE",
                "CAP_KILL"
            ],
            "ambient": [
                "CAP_NET_BIND_SERVICE"
            ]
        },
        "rlimits": [
            {
                "type": "RLIMIT_CORE",
                "hard": 1024,
                "soft": 1024
            },
            {
                "type": "RLIMIT_NOFILE",
                "hard": 1024,
                "soft": 1024
            }
        ],
        "apparmorProfile": "acme_secure_profile",
        "oomScoreAdj": 100,
        "selinuxLabel": "system_u:system_r:svirt_lxc_net_t:s0:c124,c675",
        "noNewPrivileges": true
    },
    "root": {
        "path": "rootfs",
        "readonly": true
    },
    "hostname": "slartibartfast",
    "mounts": [
        {
            "destination": "/proc",
            "type": "proc",
            "source": "proc"
        },
        {
            "destination": "/dev",
            "type": "tmpfs",
            "source": "tmpfs",
            "options": [
                "nosuid",
                "strictatime",
                "mode=755",
                "size=65536k"
            ]
        },
        {
            "destination": "/dev/pts",
            "type": "devpts",
            "source": "devpts",
            "options": [
                "nosuid",
                "noexec",
                "newinstance",
                "ptmxmode=0666",
                "mode=0620",
                "gid=5"
            ]
        },
        {
            "destination": "/dev/shm",
            "type": "tmpfs",
            "source": "shm",
            "options": [
                "nosuid",
                "noexec",
                "nodev",
                "mode=1777",
                "size=65536k"
            ]
        },
        {
            "destination": "/dev/mqueue",
            "type": "mqueue",
            "source": "mqueue",
            "options": [
                "nosuid",
                "noexec",
                "nodev"
            ]
        },
        {
            "destination": "/sys",
            "type": "sysfs",
            "source": "sysfs",
            "options": [
                "nosuid",
                "noexec",
                "nodev"
            ]
        },
        {
            "destination": "/sys/fs/cgroup",
            "type": "cgroup",
            "source": "cgroup",
            "options": [
                "nosuid",
                "noexec",
                "nodev",
                "relatime",
                "ro"
            ]
        }
    ],
    "hooks": {
        "prestart": [
            {
                "path": "/usr/bin/fix-mounts",
                "args": [
                    "fix-mounts",
                    "arg1",
                    "arg2"
                ],
                "env": [
                    "key1=value1"
                ]
            },
            {
                "path": "/usr/bin/setup-network"
            }
        ],
        "poststart": [
            {
                "path": "/usr/bin/notify-start",
                "timeout": 5
            }
        ],
        "poststop": [
            {
                "path": "/usr/sbin/cleanup.sh",
                "args": [
                    "cleanup.sh",
                    "-f"
                ]
            }
        ]
    },
    "linux": {
        "devices": [
            {
                "path": "/dev/fuse",
                "type": "c",
                "major": 10,
                "minor": 229,
                "fileMode": 438,
                "uid": 0,
                "gid": 0
            },
            {
                "path": "/dev/sda",
                "type": "b",
                "major": 8,
                "minor": 0,
                "fileMode": 432,
                "uid": 0,
                "gid": 0
            }
        ],
        "uidMappings": [
            {
                "hostID": 1000,
                "containerID": 0,
                "size": 32000
            }
        ],
        "gidMappings": [
            {
                "hostID": 1000,
                "containerID": 0,
                "size": 32000
            }
        ],
        "sysctl": {
            "net.ipv4.ip_forward": "1",
            "net.core.somaxconn": "256"
        },
        "cgroupsPath": "/myRuntime/myContainer",
        "resources": {
            "network": {
                "classID": 1048577,
                "priorities": [
                    {
                        "name": "eth0",
                        "priority": 500
                    },
                    {
                        "name": "eth1",
                        "priority": 1000
                    }
                ]
            },
            "pids": {
                "limit": 32771
            },
            "hugepageLimits": [
                {
                    "pageSize": "2MB",
                    "limit": 9223372036854772000
                }
            ],
            "memory": {
                "limit": 536870912,
                "reservation": 536870912,
                "swap": 536870912,
                "kernel": -1,
                "kernelTCP": -1,
                "swappiness": 0,
                "disableOOMKiller": false
            },
            "cpu": {
                "shares": 1024,
                "quota": 1000000,
                "period": 500000,
                "realtimeRuntime": 950000,
                "realtimePeriod": 1000000,
                "cpus": "2-3",
                "mems": "0-7"
            },
            "devices": [
                {
                    "allow": false,
                    "access": "rwm"
                },
                {
                    "allow": true,
                    "type": "c",
                    "major": 10,
                    "minor": 229,
                    "access": "rw"
                },
                {
                    "allow": true,
                    "type": "b",
                    "major": 8,
                    "minor": 0,
                    "access": "r"
                }
            ],
            "blockIO": {
                "weight": 10,
                "leafWeight": 10,
                "weightDevice": [
                    {
                        "major": 8,
                        "minor": 0,
                        "weight": 500,
                        "leafWeight": 300
                    },
                    {
                        "major": 8,
                        "minor": 16,
                        "weight": 500
                    }
                ],
                "throttleReadBpsDevice": [
                    {
                        "major": 8,
                        "minor": 0,
                        "rate": 600
                    }
                ],
                "throttleWriteIOPSDevice": [
                    {
                        "major": 8,
                        "minor": 16,
                        "rate": 300
                    }
                ]
            }
        },
        "rootfsPropagation": "slave",
        "seccomp": {
            "defaultAction": "SCMP_ACT_ALLOW",
            "architectures": [
                "SCMP_ARCH_X86",
                "SCMP_ARCH_X32"
            ],
            "syscalls": [
                {
                    "names": [
                        "getcwd",
                        "chmod"
                    ],
                    "action": "SCMP_ACT_ERRNO"
                }
            ]
        },
        "namespaces": [
            {
                "type": "pid"
            },
            {
                "type": "network"
            },
            {
                "type": "ipc"
            },
            {
                "type": "uts"
            },
            {
                "type": "mount"
            },
            {
                "type": "user"
            },
            {
                "type": "cgroup"
            }
        ],
        "maskedPaths": [
            "/proc/kcore",
            "/proc/latency_stats",
            "/proc/timer_stats",
            "/proc/sched_debug"
        ],
        "readonlyPaths": [
            "/proc/asound",
            "/proc/bus",
            "/proc/fs",
            "/proc/irq",
            "/proc/sys",
            "/proc/sysrq-trigger"
        ],
        "mountLabel": "system_u:object_r:svirt_sandbox_file_t:s0:c715,c811"
    },
    "annotations": {
        "com.example.key1": "value1",
        "com.example.key2": "value2"
    }
}

Linux Container Configuration

This document describes the schema for the Linux-specific section of the container configuration.
The Linux container specification uses various kernel features like namespaces, cgroups, capabilities, LSM, and filesystem jails to fulfill the spec.

Default Filesystems

The Linux ABI includes both syscalls and several special file paths.
Applications expecting a Linux environment will very likely expect these file paths to be set up correctly.

The following filesystems SHOULD be made available in each container's filesystem:

Path Type
/proc proc
/sys sysfs
/dev/pts devpts
/dev/shm tmpfs

Namespaces

A namespace wraps a global system resource in an abstraction that makes it appear to the processes within the namespace that they have their own isolated instance of the global resource.
Changes to the global resource are visible to other processes that are members of the namespace, but are invisible to other processes.
For more information, see the namespaces(7) man page.

Namespaces are specified as an array of entries inside the namespaces root field.
The following parameters can be specified to set up namespaces:

If a namespace type is not specified in the namespaces array, the container MUST inherit the runtime namespace of that type.
If a namespaces field contains duplicated namespaces with same type, the runtime MUST generate an error.

Example

    "namespaces": [
        {
            "type": "pid",
            "path": "/proc/1234/ns/pid"
        },
        {
            "type": "network",
            "path": "/var/run/netns/neta"
        },
        {
            "type": "mount"
        },
        {
            "type": "ipc"
        },
        {
            "type": "uts"
        },
        {
            "type": "user"
        },
        {
            "type": "cgroup"
        }
    ]

User namespace mappings

uidMappings (array of objects, OPTIONAL) describes the user namespace uid mappings from the host to the container.
gidMappings (array of objects, OPTIONAL) describes the user namespace gid mappings from the host to the container.

Each entry has the following structure:

The runtime SHOULD NOT modify the ownership of referenced filesystems to realize the mapping.
Note that the number of mapping entries MAY be limited by the kernel.

Example

    "uidMappings": [
        {
            "hostID": 1000,
            "containerID": 0,
            "size": 32000
        }
    ],
    "gidMappings": [
        {
            "hostID": 1000,
            "containerID": 0,
            "size": 32000
        }
    ]

Devices

devices (array of objects, OPTIONAL) lists devices that MUST be available in the container.
The runtime MAY supply them however it likes (with mknod, by bind mounting from the runtime mount namespace, using symlinks, etc.).

Each entry has the following structure:

The same type, major and minor SHOULD NOT be used for multiple devices.

Example

    "devices": [
        {
            "path": "/dev/fuse",
            "type": "c",
            "major": 10,
            "minor": 229,
            "fileMode": 438,
            "uid": 0,
            "gid": 0
        },
        {
            "path": "/dev/sda",
            "type": "b",
            "major": 8,
            "minor": 0,
            "fileMode": 432,
            "uid": 0,
            "gid": 0
        }
    ]

Default Devices

In addition to any devices configured with this setting, the runtime MUST also supply:

Control groups

Also known as cgroups, they are used to restrict resource usage for a container and handle device access.
cgroups provide controls (through controllers) to restrict cpu, memory, IO, pids and network for the container.
For more information, see the kernel cgroups documentation.

Cgroups Path

cgroupsPath (string, OPTIONAL) path to the cgroups.
It can be used to either control the cgroups hierarchy for containers or to run a new process in an existing container.

The value of cgroupsPath MUST be either an absolute path or a relative path.

If the value is specified, the runtime MUST consistently attach to the same place in the cgroups hierarchy given the same value of cgroupsPath.
If the value is not specified, the runtime MAY define the default cgroups path.
Runtimes MAY consider certain cgroupsPath values to be invalid, and MUST generate an error if this is the case.

Implementations of the Spec can choose to name cgroups in any manner.
The Spec does not include naming schema for cgroups.
The Spec does not support per-controller paths for the reasons discussed in the cgroupv2 documentation.
The cgroups will be created if they don't exist.

You can configure a container's cgroups via the resources field of the Linux configuration.
Do not specify resources unless limits have to be updated.
For example, to run a new process in an existing container without updating limits, resources need not be specified.

Runtimes MAY attach the container process to additional cgroup controllers beyond those necessary to fulfill the resources settings.

Example

    "cgroupsPath": "/myRuntime/myContainer",
    "resources": {
        "memory": {
        "limit": 100000,
        "reservation": 200000
        },
        "devices": [
            {
                "allow": false,
                "access": "rwm"
            }
        ]
   }

Device whitelist

devices (array of objects, OPTIONAL) configures the device whitelist.
The runtime MUST apply entries in the listed order.

Each entry has the following structure:

Example

    "devices": [
        {
            "allow": false,
            "access": "rwm"
        },
        {
            "allow": true,
            "type": "c",
            "major": 10,
            "minor": 229,
            "access": "rw"
        },
        {
            "allow": true,
            "type": "b",
            "major": 8,
            "minor": 0,
            "access": "r"
        }
    ]

Memory

memory (object, OPTIONAL) represents the cgroup subsystem memory and it's used to set limits on the container's memory usage.
For more information, see the kernel cgroups documentation about memory.

Values for memory specify the limit in bytes, or -1 for unlimited memory.

The following properties do not specify memory limits, but are covered by the memory controller:

Example

    "memory": {
        "limit": 536870912,
        "reservation": 536870912,
        "swap": 536870912,
        "kernel": -1,
        "kernelTCP": -1,
        "swappiness": 0,
        "disableOOMKiller": false
    }

CPU

cpu (object, OPTIONAL) represents the cgroup subsystems cpu and cpusets.
For more information, see the kernel cgroups documentation about cpusets.

The following parameters can be specified to set up the controller:

Example

    "cpu": {
        "shares": 1024,
        "quota": 1000000,
        "period": 500000,
        "realtimeRuntime": 950000,
        "realtimePeriod": 1000000,
        "cpus": "2-3",
        "mems": "0-7"
    }

Block IO

blockIO (object, OPTIONAL) represents the cgroup subsystem blkio which implements the block IO controller.
For more information, see the kernel cgroups documentation about blkio.

The following parameters can be specified to set up the controller:

Example

    "blockIO": {
        "weight": 10,
        "leafWeight": 10,
        "weightDevice": [
            {
                "major": 8,
                "minor": 0,
                "weight": 500,
                "leafWeight": 300
            },
            {
                "major": 8,
                "minor": 16,
                "weight": 500
            }
        ],
        "throttleReadBpsDevice": [
            {
                "major": 8,
                "minor": 0,
                "rate": 600
            }
        ],
        "throttleWriteIOPSDevice": [
            {
                "major": 8,
                "minor": 16,
                "rate": 300
            }
        ]
    }

Huge page limits

hugepageLimits (array of objects, OPTIONAL) represents the hugetlb controller which allows to limit the
HugeTLB usage per control group and enforces the controller limit during page fault.
For more information, see the kernel cgroups documentation about HugeTLB.

Each entry has the following structure:

Example

    "hugepageLimits": [
        {
            "pageSize": "2MB",
            "limit": 209715200
        }
   ]

Network

network (object, OPTIONAL) represents the cgroup subsystems net_cls and net_prio.
For more information, see the kernel cgroups documentations about net_cls cgroup and net_prio cgroup.

The following parameters can be specified to set up the controller:

Example

    "network": {
        "classID": 1048577,
        "priorities": [
            {
                "name": "eth0",
                "priority": 500
            },
            {
                "name": "eth1",
                "priority": 1000
            }
        ]
   }

PIDs

pids (object, OPTIONAL) represents the cgroup subsystem pids.
For more information, see the kernel cgroups documentation about pids.

The following parameters can be specified to set up the controller:

Example

    "pids": {
        "limit": 32771
   }

IntelRdt

intelRdt (object, OPTIONAL) represents the Intel Resource Director Technology.
If intelRdt is set, the runtime MUST write the container process ID to the <container-id>/tasks file in a mounted resctrl pseudo-filesystem, using the container ID from start and creating the <container-id> directory if necessary.
If no mounted resctrl pseudo-filesystem is available in the runtime mount namespace, the runtime MUST generate an error.

If `intelRdt` is not set, the runtime MUST NOT manipulate any `resctrl` pseudo-filesystems.

The following parameters can be specified for the container:

Example

Consider a two-socket machine with two L3 caches where the default CBM is 0xfffff and the max CBM length is 20 bits.
Tasks inside the container only have access to the "upper" 80% of L3 cache id 0 and the "lower" 50% L3 cache id 1:

"linux": {
    "intelRdt": {
        "l3CacheSchema": "L3:0=ffff0;1=3ff"
    }
}

Sysctl

sysctl (object, OPTIONAL) allows kernel parameters to be modified at runtime for the container.
For more information, see the sysctl(8) man page.

Example

    "sysctl": {
        "net.ipv4.ip_forward": "1",
        "net.core.somaxconn": "256"
   }

Seccomp

Seccomp provides application sandboxing mechanism in the Linux kernel.
Seccomp configuration allows one to configure actions to take for matched syscalls and furthermore also allows matching on values passed as arguments to syscalls.
For more information about Seccomp, see Seccomp kernel documentation.
The actions, architectures, and operators are strings that match the definitions in seccomp.h from libseccomp and are translated to corresponding values.

seccomp (object, OPTIONAL)

The following parameters can be specified to set up seccomp:

Example

    "seccomp": {
        "defaultAction": "SCMP_ACT_ALLOW",
        "architectures": [
            "SCMP_ARCH_X86",
            "SCMP_ARCH_X32"
        ],
        "syscalls": [
            {
                "names": [
                    "getcwd",
                    "chmod"
                ],
                "action": "SCMP_ACT_ERRNO"
            }
        ]
    }

Rootfs Mount Propagation

rootfsPropagation (string, OPTIONAL) sets the rootfs's mount propagation.
Its value is either slave, private, shared or unbindable.
The Shared Subtrees article in the kernel documentation has more information about mount propagation.

Example

    "rootfsPropagation": "slave",

Masked Paths

maskedPaths (array of strings, OPTIONAL) will mask over the provided paths inside the container so that they cannot be read.
The values MUST be absolute paths in the container namespace.

Example

    "maskedPaths": [
        "/proc/kcore"
    ]

Readonly Paths

readonlyPaths (array of strings, OPTIONAL) will set the provided paths as readonly inside the container.
The values MUST be absolute paths in the container namespace.

Example

    "readonlyPaths": [
        "/proc/sys"
    ]

Mount Label

mountLabel (string, OPTIONAL) will set the Selinux context for the mounts in the container.

Example

    "mountLabel": "system_u:object_r:svirt_sandbox_file_t:s0:c715,c811"

Solaris Application Container Configuration

Solaris application containers can be configured using the following properties, all of the below properties have mappings to properties specified under zonecfg(1M) man page, except milestone.

milestone

The SMF(Service Management Facility) FMRI which should go to "online" state before we start the desired process within the container.

milestone (string, OPTIONAL)

Example

"milestone": "svc:/milestone/container:default"

limitpriv

The maximum set of privileges any process in this container can obtain.
The property should consist of a comma-separated privilege set specification as described in priv_str_to_set(3C) man page for the respective release of Solaris.

limitpriv (string, OPTIONAL)

Example

"limitpriv": "default"

maxShmMemory

The maximum amount of shared memory allowed for this application container.
A scale (K, M, G, T) can be applied to the value for each of these numbers (for example, 1M is one megabyte).
Mapped to max-shm-memory in zonecfg(1M) man page.

maxShmMemory (string, OPTIONAL)

Example

"maxShmMemory": "512m"

cappedCPU

Sets a limit on the amount of CPU time that can be used by a container.
The unit used translates to the percentage of a single CPU that can be used by all user threads in a container, expressed as a fraction (for example, .75) or a mixed number (whole number and fraction, for example, 1.25).
An ncpu value of 1 means 100% of a CPU, a value of 1.25 means 125%, .75 mean 75%, and so forth.
When projects within a capped container have their own caps, the minimum value takes precedence.
cappedCPU is mapped to capped-cpu in zonecfg(1M) man page.

Example

"cappedCPU": {
        "ncpus": "8"
}

cappedMemory

The physical and swap caps on the memory that can be used by this application container.
A scale (K, M, G, T) can be applied to the value for each of these numbers (for example, 1M is one megabyte).
cappedMemory is mapped to capped-memory in zonecfg(1M) man page.

Example

"cappedMemory": {
        "physical": "512m",
        "swap": "512m"
}

Network

Automatic Network (anet)

anet is specified as an array that is used to set up networking for Solaris application containers.
The anet resource represents the automatic creation of a network resource for an application container.
The zones administration daemon, zoneadmd, is the primary process for managing the container's virtual platform.
One of the daemon's responsibilities is creation and teardown of the networks for the container.
For more information on the daemon see the zoneadmd(1M) man page.
When such a container is started, a temporary VNIC(Virtual NIC) is automatically created for the container.
The VNIC is deleted when the container is torn down.
The following properties can be used to set up automatic networks.
For additional information on properties, check the zonecfg(1M) man page for the respective release of Solaris.

Example

"anet": [
    {
        "allowedAddress": "172.17.0.2/16",
        "configureAllowedAddress": "true",
        "defrouter": "172.17.0.1/16",
        "linkProtection": "mac-nospoof, ip-nospoof",
        "linkname": "net0",
        "lowerLink": "net2",
        "macAddress": "02:42:f8:52:c7:16"
    }
]

Glossary

Bundle

A directory structure that is written ahead of time, distributed, and used to seed the runtime for creating a container and launching a process within it.

Configuration

The config.json file in a bundle which defines the intended container and container process.

Container

An environment for executing processes with configurable isolation and resource limitations.
For example, namespaces, resource limits, and mounts are all part of the container environment.

Container namespace

On Linux,the namespaces in which the configured process executes.

JSON

All configuration JSON MUST be encoded in UTF-8.
JSON objects MUST NOT include duplicate names.
The order of entries in JSON objects is not significant.

Runtime

An implementation of this specification.
It reads the configuration files from a bundle, uses that information to create a container, launches a process inside the container, and performs other lifecycle actions.

Runtime namespace

On Linux, the namespaces from which new container namespaces are created and from which some configured resources are accessed.