Upgrade instructions

If Timefold Platform is already installed, then an upgrade should be performed.

This guide assumes that the installation has been performed accordingly to the documentation about Timefold Platform, and that orbit-values.yaml file is already available.

Preparation

Depending on the target version of the platform you might need to follow preparation steps. Some might be required, some might be optional, depending on your platform installation.

0.61.0

This release comes with support for Gateway API of Kubernetes that replaces Ingress. This is optional at the moment but it is highly recommended to switch to it as soon as possible.

Install Gateway API bundle (CRDs) if not yet installed in your cluster

Follow Gateway API installation section

Cert manager configuration

If you use cert manager make sure you run on version that supports Gateway API (1.5+) and enable support for it.

helm upgrade --install cert-manager oci://quay.io/jetstack/charts/cert-manager --namespace cert-manager \
  --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \
  --set config.kind="ControllerConfiguration" \
  --set config.enableGatewayAPI=true

Install Gateway API inside your cluster

Follow Gateway implementation installation section to install NGINX Gateway Fabric and provision Gateway resource that is going to represent entry point for Timefold Platform.

Edit your values.yaml file and update ingress section

ingress:
  # disables ingress api as entry point
  enabled: false
  # and enables gateway api instead
  gateway: true

Add label to Timefold Platform namespace to be accepted by the Gateway
```
kubectl label namespace NAMESPACE_NAME gateway-access=true
```
Replace NAMESPACE_NAME with the namespace name where Timefold Platform is deployed to.
Make sure DNS is reconfigured to point to the external ip of the gateway
Upgrade Timefold Platform with regular (as used before) helm command. This will provision HTTP routes instead of ingress resources that will be bound to Gateway.
Verify that Timefold Platform is accessible via gateway and provisioned HTTP routes.
Remove Ingress controller that was previously used.

0.53.0

Added support for an additional termination setting: terminationStepCountLimit. The stepCountLimit defines the amount of steps before the solver will terminate. The terminationStepCountLimit cannot be set simultaneously with terminationUnimprovedSpentLimit.

models:
  terminationStepCountLimit: 1000

0.49.0

Support registry for model images

Support for registry for model images has been added in 0.49.0. It’s now possible to store the model images in a private registry. They can be configured by specifying the repository in the models section of the configurations.

models:
  repository: ghcr.io/timefoldai

0.42.0

Strict OIDC configuration

Strict OIDC configuration requires valid OIDC setup as it initiates the connection to OIDC provider at start. If you don’t have the proper configuration of OIDC, disable OIDC via values file:

oauth:
  enabled: false
  # certs:
  # issuer:

Disable oauth2-proxy component

oauth2-proxy component acts as OIDC proxy to allow you to initiate and manage authentication sessions. In pure headless environments this component does not bring too much value as it creates new sessions for each request anyway. In addition, when using opaque tokens it actually won’t allow authentication as it requires JWT tokens.

In situations where opaque tokens are used it is recommended to turn off the proxy component:

oauth:
  proxy: false

0.41.0 version

Added possibility to choose transport type used for map. It can be set with the following configuration:

maps:
  osrm:
    locations:
      - region: us-georgia
        transportType: car

0.40.0 version

Added possibility of configuring the max distance from road (in meters) for a map before OSRM returns no route. It can be set with the following configuration:

maps:
  osrm:
    maxDistanceFromRoad: 10000

0.39.0 version

Diminished returns termination has been added. It is used by default when no unimproved spent limit is configured. The termination is configured globally and can be overridden when submitting a problem dataset.

Optionally, values file can be extended with models configuration that allows you to set the global termination settings. Changes in this version:

terminationUnimprovedSpentLimit is now optional. When terminationUnimprovedSpentLimit is null, diminished returns termination is used (defaults to PT5M) expressed as ISO-8601 duration:

models:
  terminationSpentLimit: PT30M
  terminationMaximumSpentLimit: PT60M
  terminationUnimprovedSpentLimit:
  terminationMaximumUnimprovedSpentLimit: PT5M

It was also now possible to configure a global prefix for all buckets created by the platform, using the configuration below:

storage:
  remote:
    # Add a prefix to all created buckets (must be smaller than 28 characters)
    bucketPrefix: "prefix-timefold-buckets-"

0.36.0 version

Added option to control maximum runtime of model pods:

models:
  maximumRunningDuration: PT24H

Self-hosted environments running with a single tenant and the default plan which does not set any restrictions on max resources assigned when solving, can cause unexpected behaviour. For instance, single solver runs can take up all the memory of the node and will eventually be evicted. To prevent this at the installation level, limits can be set to restrict max amount of memory (in MB):

solver:
  # max amount of memory (in MB) assigned as limit on pod
  maxMemory: 16384

0.30.0 version

Added an option to deploy maps to an isolated node pool. See External map provider section for more details.

maps:
  isolated: true

0.28.0 version

Added an option to configure an external map provider. See External map provider section for more details.

maps:
  externalProvider: "http://localhost:5000"

0.27.0 version

Renamed the following replica count properties from Maps and Insights services: - From insightsReplicaCount, to insights.replicaCount. - From mapsServiceReplicaCount, to maps.replicaCount.

0.26.0 version

Added an option to configure external OSRM instances by location. The location should not contain any spaces:

maps:
  osrm:
    externalLocationsUrl:
      customLocation1: "http://localhost:8080"
      customLocation2: "http://localhost:8081"

0.25.0 version

Maps service component has been enhanced with scalability capabilities backed by Kubernetes StatefulSet resource. It can be enabled via maps.scalable property that needs to be set to true to deploy maps service as statefulset:

maps:
  scalable: true

Enabling scalable maps service will recreate the persistent volume currently being used. This means that any location sets stored will be removed. For that reason, the default value is set to false to keep the backward compatible mode, though it is recommended to switch to scalable mode as it allows you to run the maps service in multi-replica mode which ensures high availability and fault tolerance.

End-to-end encryption for Timefold components has been provided. See End to end encryption section for more details.

0.23.0 version

Removed the property storage.remote.expiration, that was used to configure the expiration of datasets in remote storage. This configurations is now part of each plan configuration.

Removed the property models.runOn, that was used to configure dedicated workers for solver, model, and tenant. This configurations is now part of each plan configuration.

0.22.0 version

Added default configurations for notifications webhooks read and connect timeout duration. If not set, the default is 5 seconds. These can be set with the following configuration:

notifications:
  webhook:
    connectTimeout: PT5S
    readTimeout: PT5S

0.20.0 version

Added the option to configure the maximum thread count for each job in the platform. This value is based on the maximum number of cores in the nodes of the Kubernetes cluster that are running solver jobs. It is highly recommended that this configuration is set. If not set, it defaults to 1. It can be set with the following configuration:

solver:
  maxThreadCount: 8

0.16.0 version

The location sets endpoints were changed to using hyphens instead of camel case to separate the words (/api/admin/v1/tenants/{tenantId}/maps/location-sets).
It’s possible to configure data cleanup policy to remove data sets after x days. This can be set via orbit-values.yaml file with following section:
```
storage:
  # remote storage that is used to store solver data sets
  remote:
    # specifies expiration of data sets in remote storage in days - once elapsed data will be removed. When set to 0 no expiration
    expiration: 7
```
This feature is not supported for Azure storage as it requires higher privileges to configure life cycle rules. Instead, users should configure it directly in the Azure Portal or CLI.
The configuration bucket name (used to store internal data of the platform) is now configurable. This is required to be updated for S3 and Google Cloud Storage setups as bucket names must be globally unique in those object stores. Before upgrading make sure you create uniquely named bucket (using company name or domain name) and copy all files from timefold-orbit-configuration bucket into the new one. Next, set the name of the configuration bucket in the orbit-values.yaml file under following section:
```
storage:
  remote:
    name: YOUR_CUSTOM_BUCKET_NAME
```
Replace YOUR_CUSTOM_BUCKET_NAME with the name of your choice.
The order of the locations in a location set is now not relevant to the distance calculation.

0.15.0 version

The location sets configuration for the maps cache, that was introduced in version 0.13.0, was removed from helm and it’s now done via an API (/api/admin/v1/tenants/{tenantId}/maps/locationSets).

0.14.0 version

Fine-grained control over the termination of the solver has been added. The termination is configured globally and can be overridden when submitting a problem dataset. Configuring unimproved spent limit now allows you to avoid running the solver when the score is not improving anymore. Setting maximum limits provides a safety net to prevent users from setting termination values too high.

Optionally, values file can be extended with models configuration that allows you to set the termination settings:

terminationSpentLimit: The maximum duration to keep the solver running (defaults to PT30M) expressed as ISO-8601 duration.
terminationMaximumSpentLimit: The maximum allowed limit for dataset-defined spentLimit (defaults to PT60M) expressed as ISO-8601 duration.
terminationUnimprovedSpentLimit: If the score has not improved during this period terminate the solver (defaults to PT5M) expressed as ISO-8601 duration.
terminationMaximumUnimprovedSpentLimit: The maximum allowed limit for dataset-defined unimprovedSpentLimit (defaults to PT5M) expressed as ISO-8601 duration.

models:
  terminationSpentLimit: PT30M
  terminationMaximumSpentLimit: PT60M
  terminationUnimprovedSpentLimit: PT5M
  terminationMaximumUnimprovedSpentLimit: PT5M

In case you want to opt out of the unimproved spent limit, configure it with the same value as the maximum spent limit:

models:
  terminationSpentLimit: PT30M
  terminationMaximumSpentLimit: PT60M
  terminationUnimprovedSpentLimit: PT60M
  terminationMaximumUnimprovedSpentLimit: PT60M

0.13.0 version

When using maps, it’s now possible to precalculate location sets. When doing so, the location sets will be precalculated according to a refresh interval, and will be stored in a cache by their name. For details read Deploy maps integration section.

0.12.0 version

When using OSRM maps, requests are now throttled based on the memory available in each Map. Some optional configurations were added to control the requests for the maps instances. In addition, the configurations osrmOptions and osrmLocations were replaced by osrm.options and osrm.locations, respectively. For details read Deploy maps integration section.

Optionally, values file can be extended with models configuration that allows you to set time to live for solver worker pod:

runtimeTimeToLive: The duration to keep the solver worker pod after solving has finished; expressed as ISO-8601 duration.
idleRuntimeTimeToLive: The duration to keep the solver worker pod without solving being activated, e.g. when performing recommended fit, (defaults to 10 minutes) expressed as ISO-8601 duration.

models:
  runtimeTimeToLive: PT1M
  idleRuntimeTimeToLive: PT30M

0.11.0 version

There are no required preparation steps.

0.10.0 version

When using maps there is a new component provided called maps-service that is automatically deployed when enabled via values file. In addition, when using OSRM dedicated maps can also be deployed. For details read Deploy maps integration section.

Maps service is equipped with caching layer that requires persistent volume in Kubernetes cluster for the best performance and functionality. In case persistent volumes are not available it should be disabled in values file with maps.cache.persistentVolume: null property.

0.9.0 version

There are no required preparation steps.

0.8.0 version

There are no required preparation steps.

Optionally, values file can be extended with models configuration that allows you to set default execution parameters.

models:
  terminationSpentLimit: 30m
   # IMPORTANT: when setting resources, all must be set
  resources:
    limits:
      # max CPU allowed for the platform to be used
      cpu: 1
      # max memory allowed for the platform to be used
      memory: 512Mi
    requests:
      # guaranteed CPU for the platform to be used
     cpu: 1
      # guaranteed memory for the platform to be used
     memory: 512Mi
  runOn:
    # will only run on nodes that are marked as solver workers
    solverWorkers: false
    # will only run on nodes that are marked with model id
    modelDedicated: false
    # will only run on nodes that are marked with tenant id
    tenantDedicated: false

Execution

Login to the Timefold Helm Chart repository to gain access to the Timefold Platform chart. Use user as username and token provided by Timefold support as password.

helm registry login ghcr.io/timefoldai

Once all preparation steps are completed, Timefold Platform can be upgraded with the following command:

Amazon S3 based installation

The upgrade command requires the following environment variables to be set:

NAMESPACE: The Kubernetes namespace where the platform should be installed.
OAUTH_CLIENT_SECRET: The client secret of the application used for this environment.
TOKEN: The GitHub access token to the container registry provided by Timefold support.
LICENSE: The Timefold Platform license string provided by Timefold support.
AWS_ACCESS_KEY: AWS access key.
AWS_ACCESS_SECRET: AWS access secret.

Once successfully logged in with the environment variables set, issue the upgrade command via helm:

helm upgrade --install --atomic --namespace $NAMESPACE timefold-orbit oci://ghcr.io/timefoldai/timefold-orbit-platform  \
        -f orbit-values.yaml \
        --create-namespace \
        --set namespace=$NAMESPACE \
        --set oauth.clientSecret="$OAUTH_CLIENT_SECRET" \
        --set image.registry.auth="$TOKEN" \
        --set license="$LICENSE" \
        --set secrets.data.awsAccessKey="$AWS_ACCESS_KEY" \
        --set secrets.data.awsAccessSecret="$AWS_ACCESS_SECRET"

The above command will install (or upgrade) the latest released version of the Timefold Platform. To install a specific version append --version A.B.C to the command where A.B.C is the version number to be used.

Google Cloud Storage based installation

The upgrade command requires following environment variables to be set:

NAMESPACE: Kubernetes namespace where platform should be installed.
OAUTH_CLIENT_SECRET: The client secret of the application used for this environment.
TOKEN: The GitHub access token to the container registry provided by Timefold support.
LICENSE: The Timefold Platform license string provided by Timefold support.
GCS_SERVICE_ACCOUNT: The Service Account key for Google Cloud Storage.
GCP_PROJECT_ID: The Google Cloud Project iID where storage is created.

Once successfully logged in with the environment variables set, issue the upgrade command via helm:

helm upgrade --install --atomic --namespace $NAMESPACE timefold-orbit oci://ghcr.io/timefoldai/timefold-orbit-platform  \
        -f orbit-values.yaml \
        --create-namespace \
        --set namespace=$NAMESPACE \
        --set oauth.clientSecret="$OAUTH_CLIENT_SECRET" \
        --set image.registry.auth="$TOKEN" \
        --set license="$LICENSE" \
        --set secrets.stringData.serviceAccountKey="$GCS_SERVICE_ACCOUNT" \
        --set secrets.stringData.projectId="$GCP_PROJECT_ID"

Azure BlobStore based installation

Upgrade command requires following environment variables to be set:

NAMESPACE: The Kubernetes namespace where the platform should be installed.
OAUTH_CLIENT_SECRET: The client secret of the application used for this environment.
TOKEN: The GitHub access token to the container registry provided by Timefold support.
LICENSE: The Timefold Platform license string provided by Timefold support.
AZ_CONNECTION_STRING: Azure BlobStore connection string.

Once successfully logged in with the environment variables set, issue the upgrade command via helm:

helm upgrade --install --atomic --namespace $NAMESPACE timefold-orbit oci://ghcr.io/timefoldai/timefold-orbit-platform  \
        -f orbit-values.yaml \
        --create-namespace \
        --set namespace=$NAMESPACE \
        --set oauth.clientSecret="$OAUTH_CLIENT_SECRET" \
        --set image.registry.auth="$TOKEN" \
        --set license="$LICENSE" \
        --set secrets.data.azureStoreConnectionString="$AZ_CONNECTION_STRING"