Documentation | Inference Instances › Deployments › Deprecated Create Inference Instance

API Keys

inference_instances.api_keys

Methods

Deprecated Create API Key -> { created_at, key_id, name, 6 more... }

post/v2/inference/api_keys

Deprecated. Create API Key

Deprecated Delete API Key ->

delete/v2/inference/api_keys/{key_id}

Deprecated. Delete API Key

Deprecated List API Keys ->

APIKey

get/v2/inference/api_keys

Deprecated. List API Keys

Deprecated Get API Key ->

APIKey

get/v2/inference/api_keys/{key_id}

Deprecated. Get API Key

Deprecated Update API Key ->

APIKey

patch/v2/inference/api_keys/{key_id}

Deprecated. Update API Key

Domain types

APIKey = { created_at, key_id, name, 5 more... }

Capacity

inference_instances.capacity

Methods

List Inference Flavor Capacities By Region -> { count, results }

get/v3/inference/capacity

List Inference flavor capacities by region

Deprecated Get Capacity For Regions -> { capacity, region_id }

get/v2/inference/capacity

Deprecated. Get Capacity for regions

Inference Instances Deployments

Deployments

inference_instances.deployments

Methods

Deprecated Create Inference Instance -> { id, flavor_id, image, 14 more... }

post/v2/inference/deployments

Deprecated. Create Inference Instance

Response fields

id: string

(format: uuid)

Inference instance ID.

flavor_id: string

(format: uuid)

Flavor ID for the inference instance

image: string

Docker image for the inference instance. This field should contain the image name and tag in the format 'name:tag', e.g., 'nginx:latest'. It defaults to Docker Hub as the image registry, but any accessible Docker image URL can be specified.

listening_port: number

Listening port for the inference instance.

name: string

Inference instance name.

status: "ACTIVE" | "DELETED" | "DELETING" | 7 more...

Inference instance status

address: string

Optional

(format: uri, minLength: 1)

Address of the inference instance

api_key_ids: Array<string>

Optional

List of API keys IDs attached to the inference instance

auth_enabled: boolean

Optional

Set to true if instance uses API key authentication. Manage API keys through the '/v1/inference_instances/keys' endpoint.

command: Array<string>

Optional

Command to be executed when running a container from an image.

containers: Array<{ deploy_status, region_id, scale, 1 more... }>

Optional

List of containers for the inference instance

created_at: string

Optional

(format: date-time)

Inference instance creation date in ISO 8601 format.

description: string

Optional

Inference instance description.

envs: Record<string, string>

Optional

Environment variables for the inference instance

image_registry_id: string

Optional

(format: uuid)

Image registry ID for authentication in private registries. This parameter is empty if no authentication is required for the repository.

probes: { liveness_probe, readiness_probe, startup_probe }

Optional

Probes configured for all containers of the inference instance.

timeout: number

Optional

(minimum: 0)

Specifies the duration in seconds without any requests after which the containers will be downscaled to their minimum scale value as defined by scale.min. If set, this helps in optimizing resource usage by reducing the number of container instances during periods of inactivity.

Request example

curl https://api.gcore.com/cloud/v2/inference/deployments \
    -H 'Content-Type: application/json' \
    -H "APIKey: $CLOUD_API_API_KEY" \
    -d '{
      "containers": [
        {
          "region_id": 7,
          "scale": {
            "max": 3,
            "min": 1,
            "cooldown_period": 60,
            "triggers": {
              "cpu": {
                "threshold": 80
              },
              "gpu_memory": {
                "threshold": 1
              },
              "gpu_utilization": {
                "threshold": 1
              },
              "http": {
                "rate": 1,
                "window": 1
              },
              "memory": {
                "threshold": 70
              }
            }
          }
        }
      ],
      "flavor_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
      "image": "nginx:latest",
      "listening_port": 8080,
      "name": "my-instance",
      "api_key_ids": [
        "3fa85f64-5717-4562-b3fc-2c963f66afa6"
      ],
      "auth_enabled": false,
      "command": [
        "nginx",
        "-g",
        "daemon off;"
      ],
      "description": "My first instance",
      "envs": {
        "DEBUG_MODE": "False",
        "KEY": "12345"
      },
      "image_registry_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
      "probes": {
        "liveness_probe": {
          "enabled": true,
          "probe": {
            "exec": {
              "command": [
                "ls",
                "-l"
              ]
            },
            "failure_threshold": 3,
            "http_get": {
              "port": 80,
              "headers": {
                "Authorization": "Bearer token 123"
              },
              "host": "127.0.0.1",
              "path": "/healthz",
              "schema": "HTTP"
            },
            "initial_delay_seconds": 0,
            "period_seconds": 5,
            "success_threshold": 1,
            "tcp_socket": {
              "port": 80
            },
            "timeout_seconds": 1
          }
        },
        "readiness_probe": {
          "enabled": true,
          "probe": {
            "exec": {
              "command": [
                "ls",
                "-l"
              ]
            },
            "failure_threshold": 3,
            "http_get": {
              "port": 80,
              "headers": {
                "Authorization": "Bearer token 123"
              },
              "host": "127.0.0.1",
              "path": "/healthz",
              "schema": "HTTP"
            },
            "initial_delay_seconds": 0,
            "period_seconds": 5,
            "success_threshold": 1,
            "tcp_socket": {
              "port": 80
            },
            "timeout_seconds": 1
          }
        },
        "startup_probe": {
          "enabled": true,
          "probe": {
            "exec": {
              "command": [
                "ls",
                "-l"
              ]
            },
            "failure_threshold": 3,
            "http_get": {
              "port": 80,
              "headers": {
                "Authorization": "Bearer token 123"
              },
              "host": "127.0.0.1",
              "path": "/healthz",
              "schema": "HTTP"
            },
            "initial_delay_seconds": 0,
            "period_seconds": 5,
            "success_threshold": 1,
            "tcp_socket": {
              "port": 80
            },
            "timeout_seconds": 1
          }
        }
      },
      "project_id": 1,
      "timeout": 120
    }'

200Example

{
  "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "flavor_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "image": "nginx:latest",
  "listening_port": 8080,
  "name": "my-instance",
  "status": "ACTIVE",
  "address": "https://example.com",
  "api_key_ids": [
    "3fa85f64-5717-4562-b3fc-2c963f66afa6"
  ],
  "auth_enabled": false,
  "command": [
    "nginx",
    "-g",
    "daemon off;"
  ],
  "containers": [
    {
      "deploy_status": {
        "ready": 1,
        "total": 3
      },
      "region_id": 7,
      "scale": {
        "max": 3,
        "min": 1,
        "cooldown_period": 60,
        "triggers": {
          "cpu": {
            "threshold": 80
          },
          "gpu_memory": {
            "threshold": 1
          },
          "gpu_utilization": {
            "threshold": 1
          },
          "http": {
            "rate": 1,
            "window": 1
          },
          "memory": {
            "threshold": 70
          }
        }
      },
      "error_message": "Failed to pull image"
    }
  ],
  "created_at": "2023-08-22T11:21:00Z",
  "description": "My first instance",
  "envs": {
    "DEBUG_MODE": "False",
    "KEY": "12345"
  },
  "image_registry_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "probes": {
    "liveness_probe": {
      "enabled": true,
      "probe": {
        "exec": {
          "command": [
            "ls",
            "-l"
          ]
        },
        "failure_threshold": 3,
        "http_get": {
          "port": 80,
          "headers": {
            "Authorization": "Bearer token 123"
          },
          "host": "127.0.0.1",
          "path": "/healthz",
          "schema": "HTTP"
        },
        "initial_delay_seconds": 0,
        "period_seconds": 5,
        "success_threshold": 1,
        "tcp_socket": {
          "port": 80
        },
        "timeout_seconds": 1
      }
    },
    "readiness_probe": {
      "enabled": true,
      "probe": {
        "exec": {
          "command": [
            "ls",
            "-l"
          ]
        },
        "failure_threshold": 3,
        "http_get": {
          "port": 80,
          "headers": {
            "Authorization": "Bearer token 123"
          },
          "host": "127.0.0.1",
          "path": "/healthz",
          "schema": "HTTP"
        },
        "initial_delay_seconds": 0,
        "period_seconds": 5,
        "success_threshold": 1,
        "tcp_socket": {
          "port": 80
        },
        "timeout_seconds": 1
      }
    },
    "startup_probe": {
      "enabled": true,
      "probe": {
        "exec": {
          "command": [
            "ls",
            "-l"
          ]
        },
        "failure_threshold": 3,
        "http_get": {
          "port": 80,
          "headers": {
            "Authorization": "Bearer token 123"
          },
          "host": "127.0.0.1",
          "path": "/healthz",
          "schema": "HTTP"
        },
        "initial_delay_seconds": 0,
        "period_seconds": 5,
        "success_threshold": 1,
        "tcp_socket": {
          "port": 80
        },
        "timeout_seconds": 1
      }
    }
  },
  "timeout": 120
}

Deprecated Delete Inference Instance ->

delete/v2/inference/deployments/{instance_id}

Deprecated. Delete Inference Instance

Deprecated List Inference Instances -> OffsetPage<

InferenceInstanceOut

get/v2/inference/deployments

Deprecated. List Inference Instances

Deprecated Get Inference Instance ->

InferenceInstanceOut

get/v2/inference/deployments/{instance_id}

Deprecated. Get Inference Instance

Get Inference Instance Apikey Secret -> { secret }

get/v3/inference/{project_id}/deployments/{instance_name}/apikey

Get Inference Instance Apikey Secret

Deprecated Start Inference Instance ->

post/v2/inference/deployments/{instance_id}/start

Deprecated. Start Inference Instance

Deprecated Stop Inference Instance ->

post/v2/inference/deployments/{instance_id}/stop

Deprecated. Stop Inference Instance

Deprecated Update Inference Instance ->

InferenceInstanceOut

put/v2/inference/deployments/{instance_id}

Deprecated. Update Inference Instance

Domain types

InferenceInstanceIn = { containers, flavor_id, image, 11 more... }

InferenceInstanceOut = { id, containers, flavor_id, 14 more... }

Logs

inference_instances.deployments.logs

Methods

Deprecated Get Inference Instance Logs -> { message, pod, region_id, 1 more... }

get/v2/inference/deployments/{instance_id}/logs

Deprecated. Get Inference Instance Logs

Deprecated Get Inference Instance Logs By Region -> { message, pod, time }

get/v2/inference/deployments/{instance_id}/regions/{region_id}/logs

Deprecated. Get Inference Instance Logs by Region

Flavors

inference_instances.flavors

Methods

Deprecated List Inference Instance Flavors ->

Flavor

get/v2/inference/flavors

Deprecated. List inference instance flavors

Deprecated Get Inference Instance Flavor Details ->

Flavor

get/v2/inference/flavors/{flavor_id}

Deprecated. Get inference instance flavor Details

Domain types

Flavor = { id, cpu, description, 5 more... }

Models

inference_instances.models

Methods

List Ml Model Catalog ->

MlModelCatalog

get/v2/inference/models

List ML Model Catalog

Get Ml Model Catalog Details ->

MlModelCatalog

get/v2/inference/models/{model_id}

Get ML Model Catalog Details

Domain types

MlModelCatalog = { id, description, image_url, 15 more... }

Registries

inference_instances.registries

Methods

Deprecated Create Registry ->

post/v2/inference/registries

Deprecated. Create Registry

Deprecated Delete Registry ->

delete/v2/inference/registries/{registry_id}

Deprecated. Delete Registry

Deprecated List Registries ->

get/v2/inference/registries

Deprecated. List Registries

Deprecated Get Registry ->

get/v2/inference/registries/{registry_id}

Deprecated. Get Registry

Deprecated Update Registry ->

patch/v2/inference/registries/{registry_id}

Deprecated. Update Registry

Domain types

ImageRegistry = { id, created_at, name, 2 more... }

Registry Credentials

inference_instances.registry_credentials

Methods

Create Inference Registry Credential -> { name, password, project_id, 2 more... }

post/v3/inference/{project_id}/registry_credentials

Create Inference Registry Credential

Delete Inference Registry Credential ->

delete/v3/inference/{project_id}/registry_credentials/{credential_name}

Delete Inference Registry Credential

List Inference Registry Credentials -> OffsetPage<{ name, project_id, registry_url, 1 more... }>

get/v3/inference/{project_id}/registry_credentials

List Inference Registry Credentials

Get Inference Registry Credential -> { name, project_id, registry_url, 1 more... }

get/v3/inference/{project_id}/registry_credentials/{credential_name}

Get Inference Registry Credential

Update Inference Registry Credential ->

put/v3/inference/{project_id}/registry_credentials/{credential_name}

Update Inference Registry Credential

Secrets

inference_instances.secrets

Methods

Create Inference Secret ->

post/v3/inference/{project_id}/secrets

Create Inference Secret

Delete Inference Secret ->

delete/v3/inference/{project_id}/secrets/{secret_name}

Delete Inference Secret

List Secrets For Inference -> OffsetPage<

get/v3/inference/{project_id}/secrets

List Secrets for Inference

Get Inference Secret ->

get/v3/inference/{project_id}/secrets/{secret_name}

Get Inference Secret

Update Inference Secret ->