Running Batch Changes Server Side

Learn how to run Batch Changes server-side using executors and file mounts.

By default, Batch Changes uses a command line interface in your local environment to compute diffs and create changesets. This can be impractical for creating batch changes affecting hundreds or thousands of repositories, with large numbers of workspaces, or if the batch change steps require CPU, memory, or disk resources that are unavailable locally.

Instead of computing Batch Changes locally using src-cli, you can offload this task to one or many remote server called an executor. Executors are also required to enable code navigation auto-indexing.

This allows to:

• Run large-scale batch changes that would be impractical to compute locally
• Speed up batch change creation time by distributing batch change computing over several executors
• Reduce the setup time required to onboard new users to Batch Changes

Setup

This is a one-time process. Once a site-admin of the Sourcegraph instance sets up executors and enables running batch changes server-side, all users of the Sourcegraph instance can get started with no additional setup required.

Make sure that executors are deployed and are online.

Limitations

• Running batch changes server-side requires setting up executors. Executors are configured ready-to-use on Sourcegraph Cloud
• Running batch changes server-side is limited to user namespaces
• The newly introduced APIs for server-side are still experimental and will likely change
• Executors can only be deployed using Terraform (AWS or GCP) or using pre-built binaries (see deploying executors).

Running batch changes server-side has been tested to run a simple 45K changeset batch change. Actual performance and setup requirements depend on the complexity of the batch change.

Note for Apple Silicon Mac users

By default, docker on mac will build docker images for linux/arm64, which will result in errors when running server-side because executors provide linux/amd64 hosts. If you're creating your own images to run in batch changes, this can be a problem. Use the --platform linux/amd64 flag with docker build to build images compatible with the server-side host.

Using file mounts with server-side execution

File mounts are a powerful way to run custom files without directly embedding the files in your batch spec.

Writing a batch spec

In the following example, you have a Python script that appends "Hello World" to all README files.

PYTHON
#!/usr/bin/env python3
import os.path
 
 
def main():
  if os.path.exists('README'):
    with open('README', 'a') as f:
      f.write('\nHello World')
 
 
if __name__ == "__main__":
  main()

To use the Python script in your batch change, mount the script in a step using the mounts field. The following is an example of mounting the above Python script in a step.

YAML
name: hello-world
description: Add Hello World to READMEs
 
# Find all repositories that contain a README file.
on:
  - repositoriesMatchingQuery: file:README
 
# In each repository, run this command. Each repository's resulting diff is captured.
steps:
  - run: python /tmp/hello_appender.py
    container: python:latest
    mount:
      - path: ./hello_appender.py
        mountpoint: /tmp/hello_appender.py
 
# Describe the changeset (e.g., GitHub pull request) you want for each repository.
changesetTemplate:
  title: Hello World
  body: My first batch change!
  branch: hello-world # Push the commit to this branch.
  commit:
    message: Append Hello World to all README files
  published: false # Do not publish any changes to the code hosts yet

In this example, the Python script should live beside the batch spec file, as indicated by the path:

TEXT
.
├── batch-spec.yml
└── hello_appender.py

Note that a container appropriate for the mounted file has also been chosen for this step.

Running server-side

After writing the batch spec, use the Sourcegraph CLI (src) command remote to execute the batch spec server-side.

SHELL
src batch remote -f batch-spec.yml

Once successful, src provides a URL to the execution of the batch change.