Workload API

• 1: Pod Group Disruption and Priority
• 2: PodGroup Scheduling Policies
• 3: Topology-Aware Workload Scheduling
• 4: Scheduling Building Block APIs and the workloadbuilder Library

apiVersion: scheduling.k8s.io/v1beta1
kind: Workload
metadata:
  name: training-job-workload
  namespace: some-ns
spec:
  controllerRef:
    apiGroup: batch
    kind: Job
    name: training-job
  podGroupTemplates:
  - name: workers
    schedulingPolicy:
      gang:
        # The gang is schedulable only if 4 pods can run at once
        minCount: 4
    priorityClassName: high-priority
    disruptionMode:
      all: {}

apiVersion: scheduling.k8s.io/v1alpha3
kind: Workload
metadata:
  name: gang-of-gangs-workload
  namespace: default
spec:
  compositePodGroupTemplates:
  - name: root
    schedulingPolicy:
      gang:
        # Requires both child PodGroups to be schedulable together
        minGroupCount: 2
    podGroupTemplates:
    - name: workers-a
      schedulingPolicy:
        gang:
          # Requires 4 Pods in this group to be schedulable
          minCount: 4
    - name: workers-b
      schedulingPolicy:
        gang:
          # Requires 4 Pods in this group to be schedulable
          minCount: 4

1 - Pod Group Disruption and Priority

PodGroup can declare a disruption mode. This mode dictates how the scheduler can disrupt a running PodGroup, for example to accommodate a higher priority PodGroup. A PodGroup also has a priority, which overrides the priority of the individual pods from the group for workload-aware preemption events.

Disruption mode types

The API supports two disruption modes: Single and All. The default one is Single.

Single

The Single mode instructs the scheduler to treat all Pods in the group as separate entities, allowing independent disruption of a single pod from a PodGroup.

All

The All mode emphasizes "all-or-nothing" semantics for disruption. It instructs the scheduler that all pods from the PodGroup have to be disrupted together.

CompositePodGroup

A CompositePodGroup can also declare a disruptionMode in its specification, which controls how the scheduler disrupts child groups within the composite group during preemption events.

The API supports two disruption modes for CompositePodGroups:

• Single: Allows individual child groups within the CompositePodGroup to be disrupted independently during preemption.
• All: Enforces all-or-nothing disruption semantics across the CompositePodGroup hierarchy. If any Pod contained in the hierarchy below this CompositePodGroup has to be preempted, all of the Pods from the entire hierarchy must be preempted.

If not specified, the mode defaults to Single.

Pod group priority

PodGroup uses the same concept of PriorityClass as single Pods. Once you have created one or more PriorityClasses, you can create a PodGroup that specifies one of those PriorityClass names in its specification. The priority admission controller uses the priorityClassName field and populates the integer value of the priority. If the priority class is not found, the PodGroup is rejected. When priorityClassName is not set for a PodGroup, Kubernetes looks for a default (a PriorityClass with globalDefault set true) If there is no PriorityClass with globalDefault set true, a PodGroup with no priorityClassName has priority zero.

The priority of the PodGroup is an authoritative priority for all pods in the group during workload-aware preemption events. This value is also used for the ordering of PodGroups in the scheduling queue. When the priorities of individual pods forming this PodGroup differ from PodGroup priority the PodGroup will not be scheduled with all pods in a single pod group should have the same priority as the pod group error.

When the PodGroupPreemptionPolicy feature gate is enabled, PodGroup has also preemptionPolicy field. This field is also taken from the PriorirtyClass. It is an authoratitive field for all pods in the group and it decides whether the PodGroup can perform a preemption of lower priority pods and pod groups to accomodate a place for itself. When the feature gate is enabled all pods in the PodGroup must have the same preemptionPolicy as PodGroup. Otherwise the PodGroup will not be scheduled with all pods in a single pod group should have the same preemption policy as the pod group's preemption policy error. When PodGroup has preemptionPolicy: Never it will not perform workload aware preemption. If the feature flag is disabled, all pods forming PodGroup must have the same preemptionPolicy. Otherwise the PodGroup will not be scheduled with all pods in a single pod group should have the same preemption policy error.

The following YAML is an example of a PodGroup configuration that uses the high-priority PriorityClass, which maps to the integer priority value of 1000000. The priority admission controller checks the specification and resolves the priority of the PodGroup to 1000000.

apiVersion: scheduling.k8s.io/v1beta1
kind: PodGroup
metadata:
  namespace: ns-1
  name: job-1
spec:
  priorityClassName: high-priority

CompositePodGroup priority

CompositePodGroup API has priorityClassName and priority fields as well and their resolution is performed in the same way as for the PodGroups, through the priority admission controller.

The priority of a root CompositePodGroup acts as the authoritative priority for all child groups and Pods within its hierarchy during workload-aware preemption events. All Pods within a single group hierarchy must share the exact same priority and must be equal to the priority of the root CompositePodGroup.

The value of priority is also used for the ordering of root CompositePodGroups in the scheduling active queue.

PreemptionPolicy in CompositePodGroup

The CompositePodGroup API has the preemptionPolicy field as well and its resolution is performed in the exact same way as for the PodGroup API.

The value of preemptionPolicy of the root CompositePodGroup determines whether workload-aware preemption can be invoked to fit its Pods during scheduling if needed:

• PreemptLowerPriority policy allows preempting victims with lower priority,
• Never policy disables workload-aware preemption for that root CompositePodGroup.

All Pods within a single group hierarchy must share the exact same preemption policy which must be equal to the preemption policy of the root CompositePodGroup.

If the feature flag is disabled, the root CompositePodGroup will be allowed to perform preemption unless one of the Pods that belongs to the group hierarchy has preemptionPolicy set to Never.

What's next

• Read about Workload-Aware Preemption algorithm.
• Learn about the Workload API.
• Learn about the scheduling building blocks and the workloadbuilder library, including the disruption mode building block.

2 - PodGroup Scheduling Policies

Every PodGroup must declare a scheduling policy in its spec.schedulingPolicy field. This policy dictates how the scheduler treats the collection of Pods in the group.

Policy types

The schedulingPolicy field supports two policy types: basic and gang. You must specify exactly one.

Basic policy

The basic policy instructs the scheduler to evaluate all Pods on a best-effort basis. Unlike the gang policy, a PodGroup using the basic policy is considered feasible regardless of how many of its Pods are currently schedulable.

The primary reason to use the basic policy is to organize Pods into a group for better observability and management, while still evaluating them together within a single, atomic PodGroup scheduling cycle.

This policy is suited for groups that do not require simultaneous startup but logically belong together, or to open the way for group-level constraints that do not imply "all-or-nothing" placement.

schedulingPolicy:
  basic: {}

Gang policy

The gang policy enforces "all-or-nothing" scheduling. This is essential for tightly-coupled workloads where partial startup results in deadlocks or wasted resources.

This can be used for Jobs or any other batch process where all workers must run concurrently to make progress.

The gang policy requires a minCount field, which is the minimum number of Pods that must be schedulable simultaneously for the group to be feasible:

schedulingPolicy:
  gang:
    # The number of Pods that must be schedulable simultaneously
    # for the group to be admitted.
    minCount: 4

Setting policies via PodGroupTemplates

When using the Workload API, you define scheduling policies inside PodGroupTemplates. The workload controller copies the policy from the template into each PodGroup it creates, making the PodGroup self-contained. Changes to the Workload's templates only affect newly created PodGroups, not existing ones.

For standalone PodGroups (created without a Workload), you set spec.schedulingPolicy directly on the PodGroup itself.

Policies in CompositePodGroups

When the CompositePodGroup feature gate and the scheduling.k8s.io/v1alpha3 API group are enabled, CompositePodGroupTemplates in a Workload and the CompositePodGroup objects also declare a scheduling policy.

While a scheduling policy in a PodGroup governs a collection of individual Pods, a CompositePodGroup scheduling policy governs its direct child groups (which can be both CompositePodGroup and PodGroup objects).

Policy types for CompositePodGroups

Similar to PodGroups, the spec.schedulingPolicy field of a CompositePodGroup supports two types:

• basic: Child groups within the CompositePodGroup are evaluated and admitted independently.
• gang: Enforces multi-level all-or-nothing scheduling across child groups. The CompositePodGroup is schedulable only if at least minGroupCount child groups can be scheduled simultaneously.

schedulingPolicy:
  gang:
    # The minimum number of child groups that must be schedulable
    # simultaneously for this composite group to be admitted.
    minGroupCount: 2

Setting composite policies via templates

When using the Workload API, scheduling policies for CompositePodGroups are defined inside CompositePodGroupTemplates. Workload controllers copy the schedulingPolicy specified in the templates into each CompositePodGroup created at runtime. Unlike leaf PodGroupTemplates where minCount can be updated, minGroupCount in a CompositePodGroupTemplate is immutable.

What's next

• See the PodGroup API for how policies are carried at runtime.
• Learn about the Workload API that defines PodGroupTemplates.
• Read about PodGroup scheduling.
• Read about the gang scheduling algorithm.
• Learn about the scheduling building blocks and the workloadbuilder library that controllers use to compile these policies.

3 - Topology-Aware Workload Scheduling

Topology-Aware Scheduling (TAS) is a feature of the Workload API that optimizes the placement of pods within the cluster.

TAS ensures that all pods within a PodGroup are co-located into a specific topology domain, such as a single server rack or zone. This minimizes inter-pod communication latency and prevents workload fragmentation across the cluster infrastructure.

Topology-aware scheduling with gang scheduling policy

When applied to PodGroups with gang scheduling policy, TAS simulates the potential assignment (placement) of the full group of pods at once. It guarantees that at least the specified minCount pods can fit together into the same topology domain before committing resources. If no feasible placement is found, the entire PodGroup becomes unschedulable.

This is the recommended approach for workloads like distributed AI and ML training that strictly require proximity to minimize inter-pod communication latency.

If new pods are added to the PodGroup where some pods are already scheduled (for example, if pods are recreated), the scheduler will force all new incoming pods to land on the exact same topology domain where the existing pods currently reside. If that specific domain lacks sufficient capacity for the new pods, the pods will remain pending - even if it means that less than minCount pods are scheduled at this point.

Topology-aware scheduling with basic scheduling policy

Using TAS with basic scheduling policy may exhibit inconsistent behavior. The scheduler may only observe a subset of pods when entering the PodGroup scheduling cycle - therefore placement feasibility is only evaluated for the observed pods, rather than the entire PodGroup. To partially mitigate this limitation, you can use scheduling gates to hold off PodGroup scheduling until all pods within the PodGroup are in the scheduling queue.

If no feasible placement is found for the entire PodGroup, only a subset of pods may be scheduled, and they are guaranteed to meet the scheduling constraints.

If new pods are added to the PodGroup where some pods are already scheduled, the scheduler will act the same as in case of gang policy - forcing the new pods into the same domain, unless there is insufficient capacity (in which case the new pods will remain pending).

API configuration: scheduling constraints

Every PodGroup (or PodGroupTemplate) may optionally declare the schedulingConstraints field, which is interpreted by the placement-based PodGroup scheduling algorithm. If constraints are defined in PodGroupTemplate, they will be copied to referencing PodGroups.

As of Kubernetes v1.36, the API supports topology constraints.

Topology constraint

To define a topology constraint for a PodGroup you need to set a key, which corresponds to a Kubernetes node label, representing the target topology domain (for example, a rack or a zone). The scheduler strictly enforces that all pods within the PodGroup are placed onto nodes that share the exact same value for this specified label.

Here is an example of a PodGroup configured with a topology constraint:

apiVersion: scheduling.k8s.io/v1beta1
kind: PodGroup
metadata:
  name: example-podgroup
spec:
  schedulingPolicy:
    gang:
      minCount: 4
  schedulingConstraints:
    topology:
      - key: topology.example.com/rack

Multi-level topology-aware scheduling

Complex workloads might require co-location of their Pods at different levels of the cluster infrastructure. For example, an entire workload may need to run within a single availability zone, while different parts of that workload may require strict co-location within specific server racks.

Such multi-level co-location requirements can be expressed using the CompositePodGroup API and by specifying topology constraints at different levels of a group hierarchy.

Using the CompositePodGroup API requires enabling the CompositePodGroup feature gate and the scheduling.k8s.io/v1alpha3 API group.

Multi-level topology constraints resolution

Every group inside a CompositePodGroup hierarchy can specify a topology constraint which guarantees that all descendant Pods of that group will be scheduled in the same topology domain, matching that group's constraint.

During hierarchical scheduling, the scheduler resolves these constraints in a top-down manner. Specifically, topology domains that are considered during scheduling of a child group are confined within a topology domain that corresponds to the placement assumed by the parent group.

Kubernetes does not impose any strict requirements on the physical hierarchy of topology labels - topology keys are arbitrary node labels. However, the order in which you specify topology constraints from parent to child determines the order in which the scheduler subdivides topology domains.

Example

The following example configures a Workload where the parent CompositePodGroupTemplate constrains the entire workload to a single availability zone (topology.example.com/zone), while two child PodGroupTemplate entries (workers and driver) constrain their respective Pods to server racks (topology.example.com/rack) within that zone:

apiVersion: scheduling.k8s.io/v1alpha3
kind: Workload
metadata:
  name: example-workload
spec:
  compositePodGroupTemplates:
  - name: root
    schedulingPolicy:
      gang:
        minGroupCount: 2
    schedulingConstraints:
      topology:
      - key: topology.example.com/zone
    podGroupTemplates:
    - name: workers
      schedulingPolicy:
        gang:
          minCount: 8
      schedulingConstraints:
        topology:
        - key: topology.example.com/rack
    - name: driver
      schedulingPolicy:
        gang:
          minCount: 1
      schedulingConstraints:
        topology:
        - key: topology.example.com/rack

After creating the Workload object, the corresponding group objects are created as follows:

• Root CompositePodGroup referencing the root template.
• Two child PodGroup objects (workers and driver), each referencing the root CompositePodGroup as their parent group.

apiVersion: scheduling.k8s.io/v1alpha3
kind: CompositePodGroup
metadata:
  name: workload-root
spec:
  workloadRef:
    workloadName: example-workload
    templateName: root
  schedulingPolicy:
    gang:
      minGroupCount: 2
  schedulingConstraints:
    topology:
    - key: topology.example.com/zone
---
apiVersion: scheduling.k8s.io/v1beta1
kind: PodGroup
metadata:
  name: workload-workers
spec:
  parentCompositePodGroupName: workload-root
  workloadRef:
    workloadName: example-workload
    templateName: workers
  schedulingPolicy:
    gang:
      minCount: 8
  schedulingConstraints:
    topology:
    - key: topology.example.com/rack
---
apiVersion: scheduling.k8s.io/v1beta1
kind: PodGroup
metadata:
  name: workload-driver
spec:
  parentCompositePodGroupName: workload-root
  workloadRef:
    workloadName: example-workload
    templateName: driver
  schedulingPolicy:
    gang:
      minCount: 1
  schedulingConstraints:
    topology:
    - key: topology.example.com/rack

During scheduling, the scheduler first selects an availability zone for workload-root. It then subdivides the nodes in that zone by rack to find feasible rack placements for workload-workers and workload-driver within the selected zone.

For example, consider a cluster with five nodes labeled as follows:

When processing workload-root, the scheduler evaluates candidate placements across all cluster nodes based on the topology.example.com/zone topology key:

When evaluating candidate placements for workload-workers, the scheduler subdivides only the nodes within the placement assumed by workload-root based on the topology.example.com/rack topology key:

Candidate placements generated for the sibling workload-driver PodGroup are identical to those generated for workload-workers, since both groups specify the same topology key (topology.example.com/rack).

What's next

• Learn about pod group policies.
• Learn about plugins related Topology-aware Scheduling
• Read about gang scheduling algorithm.
• Learn about the scheduling building blocks and the workloadbuilder library, including the scheduling constraints building block.

4 - Scheduling Building Block APIs and the workloadbuilder Library

Workload-aware Scheduling defines a set of reusable API building blocks under the scheduling.k8s.io API group. Controller authors embed these primitives into their own APIs so that users express scheduling intent (gang scheduling, topology, disruption behavior) with a consistent schema across the ecosystem, and the shared workloadbuilder library compiles that intent into the scheduler-facing Workload, PodGroup, and CompositePodGroup objects.

The built-in consumer of these building blocks today is the Job controller, gated by the WorkloadWithJob feature gate.

Reusable building blocks

The building blocks are strongly-typed Go structs in the scheduling.k8s.io/v1alpha3 API group. They are meant for controller authors: a controller embeds these structs into its own API type as fields, and the workloadbuilder library compiles them. The type names follow two conventions: leaf-level types are prefixed WorkloadPodGroup... (for example, WorkloadPodGroupSchedulingPolicy) and the multi-level variants - WorkloadCompositePodGroup....

Each controller chooses the field names and structure that are idiomatic for its own API, so these blocks impose no fixed top-level shape. Although not strictly enforced, reusing the standard field names and structure is recommended, so that a user who has configured gang scheduling on one controller's resource recognizes the same options on another's.

To adopt the building blocks, a controller adds the ones it wants to support as fields on its own types. For example, the Job API groups all four of them into a single type reachable at spec.scheduling:

type JobSchedulingConfiguration struct {
    SchedulingPolicy      *schedulingv1alpha3.WorkloadPodGroupSchedulingPolicy
    SchedulingConstraints *schedulingv1alpha3.WorkloadPodGroupSchedulingConstraints
    DisruptionMode        *schedulingv1alpha3.WorkloadPodGroupDisruptionMode
    ResourceClaims        []schedulingv1alpha3.WorkloadPodGroupResourceClaim
}

A different controller might nest the same blocks per component of a multi-part workload, or support only a subset of them.

Scheduling policy

The scheduling policy block carries the same basic and gang policies as a PodGroup's spec.schedulingPolicy. See PodGroup scheduling policies for what each policy means and how the scheduler applies it.

The one difference is that the block's gang minimum count is optional. Users may leave minCount unset, in which case the controller supplies a default that makes sense for its own domain; the Job controller, for example, uses the Job's parallelism.

Scheduling constraints

The scheduling constraints block carries the topology constraints documented in Topology-aware workload scheduling: a node label key naming the domain (such as a rack or a zone) that every Pod in the group must share, with at most one topology constraint per group. Controllers should freeze the field after creation, since constraints are immutable in the compiled Workload.

Disruption mode

The disruption mode block selects whether the group's Pods may be disrupted individually (single) or only as a unit (all), corresponding to the Pod and PodGroup disruption modes documented in Pod group disruption and priority.

The library rejects combinations that are not meaningful. For example, prevent all disruption mode for PodGroups with BasicSchedulingPolicy, because the preemption unit must not be larger than the scheduling unit - a group scheduled pod-by-pod has no group-level unit to preempt or disrupt.

Resource claims

The resource claims block expresses which dynamic resource allocation claims are shared by every Pod in the group rather than allocated per Pod. Each entry names the claim within the group and points at either an existing ResourceClaim or a ResourceClaimTemplate from which one is generated. A group may declare at most four claims.

Pods consume the devices allocated to the group by declaring a matching claim in their own spec, using the same name and referring to the same object.

Composite building blocks

Multi-level controllers that orchestrate other controllers (for example, JobSet creating Jobs) coordinate a group of groups. For that layer, the API provides an analogous set of primitives prefixed with WorkloadCompositePodGroup... (for example WorkloadCompositePodGroupSchedulingPolicy). They follow the same shapes as the leaf-level blocks, except the composite gang policy uses minGroupCount (the minimum number of child groups that must be schedulable together) in place of the leaf's minCount. Keeping leaf and composite types distinct lets each hierarchy level evolve independently.

Example: the Job integration

The Job controller is the built-in example of these blocks in use. A user fills in a Job's spec.scheduling, and the controller compiles it into a Workload and PodGroup. See Integrate with Workload APIs for a complete Job manifest, the defaults that apply when spec.scheduling is omitted, and which fields you can change after creation.

The workloadbuilder library

workloadbuilder is a shared Go library that turns a controller's scheduling intent into the scheduler-facing Workload and its runtime PodGroup/CompositePodGroup objects, so each controller does not reimplement defaulting, validation, and template compilation. It is designed for both in-tree controllers (such as the Job controller) and out-of-tree controllers (such as JobSet or Kubeflow TrainJob), which vendor it like any other Go dependency. It ships from k8s.io/component-helpers/scheduling/schedulingv1/workloadbuilder.

The library consumes the scheduling.k8s.io/v1alpha3 building blocks and compiles them into scheduling.k8s.io/v1beta1 Workload and PodGroup objects, while CompositePodGroup objects remain scheduling.k8s.io/v1alpha3.

How a controller uses it

A controller describes its workload as a tree of WorkloadItem nodes, one per logical component. A node with no children becomes a single PodGroupTemplate, while a node with children becomes a CompositePodGroupTemplate over them, which is how a multi-level controller represents a group of groups. Each node carries:

• a default config, the controller's own defaults for anything the user leaves unset. This is where a controller decides, for example, that an unconfigured Job stays on basic scheduling.
• an input, the user's intent taken from the controller's API. The controller records each building block together with the field path it lives at, so validation errors point at the exact field the user set.
• optional callbacks, which adjust the merged configuration. This is how a controller supplies a context-specific default, such as filling in an unset gang minimum count from the Job's parallelism.

The controller then hands that tree to a Builder and works through four calls:

1. NewBuilder constructs the builder from the tree, along with the name, namespace, and owner reference for the object to be produced. The owner becomes the Workload's controller reference, which is used for discovery and garbage collection.
2. Validate resolves the tree and reports any problems as a list of field errors, which a controller returns from its own API validation.
3. BuildWorkload compiles the tree into a Workload. The result is cached, so several PodGroups can be created from one compiled result.
4. NewPodGroup creates a runtime PodGroup from one of the compiled templates, naming the template it should be built from.

builder := workloadbuilder.NewBuilder(item, opts)
if errs := builder.Validate(ctx, workloadbuilder.ValidationInput{}); len(errs) > 0 {
    // reject the request
}
workload, err := builder.BuildWorkload()
podGroup, err := builder.NewPodGroup("trainer-pg", item.Name)

For complete, runnable versions of this flow, including how the Job controller wires it up, see the examples in the package reference.

Validating a scheduling configuration

Validate checks a configuration in two layers:

• Structural validation of the building blocks themselves: required fields, value ranges, the rule that exactly one member of a union is set, and immutability. These checks come from declarative validation rules generated from the API types.
• Controller-policy checks that declarative validation cannot express: the allow-lists described below, and cross-field rules such as rejecting the all disruption mode alongside the basic policy.

Validation also differs between creating and updating an object. On an update, the library additionally enforces the fields that are frozen after creation, which means the controller has to supply the previously stored configuration along with the new one. On a create there is nothing to compare against, so those checks do not apply.

Opting out of declarative validation

Whether you want the first layer depends on where your controller runs, and one option controls it:

• Out-of-tree controllers leave declarative validation enabled, which is the default. Nothing else applies those structural rules to a custom resource, so a single Validate call covers both layers.
• In-tree controllers set DisableDeclarativeValidation, because the API server already runs declarative validation on the embedded blocks while validating the parent object. Skipping the first layer avoids checking the same fields twice, leaving Validate to run only the controller-policy checks.

Opting in to scheduling options

Because the building-block types are shared across controllers, future releases may add scheduling options that do not make sense for every controller. To keep new options from silently leaking in, workloadbuilder uses an allow-list model: a controller declares the policies and disruption modes it supports, and Validate rejects anything outside that set, reporting the error at the offending block's field path.

Options are therefore denied by default. When a new policy is introduced, an existing controller keeps rejecting it until its maintainers extend the allow-list, which for an out-of-tree controller means updating its vendored copy of the library as well.

builder := workloadbuilder.NewBuilder(item, workloadbuilder.BuildOptions{
    Owner:                  owner,
    AllowedPolicies:        []workloadbuilder.SchedulingPolicyOption{workloadbuilder.BasicPolicy, workloadbuilder.GangPolicy},
    AllowedDisruptionModes: []workloadbuilder.DisruptionModeOption{workloadbuilder.SingleMode, workloadbuilder.AllMode},
})
allErrs := builder.Validate(ctx, workloadbuilder.ValidationInput{})

Generating PodGroups from an existing Workload

When the Workload already exists, whether compiled by a parent controller or created manually, a child controller that only manages the runtime PodGroup uses NewBuilderFromExistingWorkload instead. That builder creates PodGroup objects from the supplied Workload using its own owner reference. It does not validate or compile anything, so the existing Workload is never recompiled.

builder := workloadbuilder.NewBuilderFromExistingWorkload(parentWorkload, workloadbuilder.BuildOptions{Owner: owner})
podGroup, err := builder.NewPodGroup("trainer-pg", "trainer-pgt-0")

What's next

• Learn about PodGroup scheduling policies.
• Learn about Pod group disruption and priority.
• Learn about Topology-aware workload scheduling.
• See the Workload API overview.
• See how the Job controller integrates with the Workload APIs.
• Read the workloadbuilder package reference.