Numba Release Process

The goal of the Numba release process – from a high level perspective – is to publish source and binary artifacts that correspond to a given version number. This usually involves a sequence of individual tasks that must be performed in the correct order and with diligence. Numba and llvmlite are commonly released in lockstep since there is usually a one-to-one mapping between a Numba version and a corresponding llvmlite version.

This section contains various notes and templates that can be used to create a Numba release checklist on the Numba Github issue tracker. This is an aid for the maintainers during the release process and helps to ensure that all tasks are completed in the correct order and that no tasks are accidentally omitted.

If new or additional items do appear during release, please do remember to add them to the checklist templates. Also note that the release process itself is always a work in progress. This means that some of the information here may be outdated. If you notice this please do remember to submit a pull-request to update this document.

All release checklists are available as Gitub issue templates. To create a new release checklist simply open a new issue and select the correct template.

Primary Release Candidate Checklist

This is for the first/primary release candidate for minor release i.e. the first release of every series. It is special, because during this release, the release branch will have to be created. Release candidate indexing begins at 1.

## Numba X.Y.Z

* [ ] Merge to main.
    - [ ] "remaining Pull-Requests from milestone".
* [ ] Check Numba's version support table documentation. Update via PR if
      needed.
* [ ] Review deprecation schedule and notices. Make PRs if need be.
* [ ] Merge change log changes.
    - [ ] "PR with changelog entries".
* [ ] Create X.Y release branch.
* [ ] Create PR against the release branch to make `numba/testing/main.py`
      to refer to `origin/releaseX.Y` instead of `origin/main`.
* [ ] Dependency version pinning on release branch
  * [ ] Pin llvmlite to `0.A.*`.
  * [ ] Pin NumPy if needed
  * [ ] Pin TBB if needed
* [ ] Run the HEAD of the release branch through the build farm and confirm:
  * [ ] Build farm CPU testing has passed.
  * [ ] Build farm CUDA testing has passed.
  * [ ] Build farm wheel testing has passed.
* [ ] Annotated tag `X.Y.Zrc1` on release branch (no `v` prefix).
* [ ] Build and upload conda packages on buildfarm (check "upload").
* [ ] Build wheels and sdist on the buildfarm (check "upload").
* [ ] Verify packages uploaded to Anaconda Cloud and move to `numba/label/main`.
* [ ] Upload wheels and sdist to PyPI (upload from `ci_artifacts`).
* [ ] Verify wheels for all platforms arrived on PyPi.
* [ ] Initialize and verify ReadTheDocs build.
* [ ] Post announcement to discourse group and ping the release testers group
  using `@RC_Testers`.
* [ ] Post link to X and to Mastodon and...

### Post Release:

* [ ] Clean up `ci_artifacts` by moving files to sub-directories
* [ ] Tag `X.Y+1.0dev0` to start new development cycle on `main`.
* [ ] Update llvmlite dependency via PR to `main`, PR includes version updates
      to:
  * [ ] `setup.py`
  * [ ] `numba/__init__.py`
  * [ ] `docs/environment.yml`
  * [ ] `buildscripts/incremental/setup_conda_environment.sh`
  * [ ] `buildscripts/incremental/setup_conda_environment.cmd`
  * [ ] `buildscripts/condarecipe.local/meta.yaml`
* [ ] Update release checklist template with any additional bullet points that
      may have arisen during the release.
* [ ] Close milestone (and then close this release issue).

Open a primary release checklist.

Subsequent Release Candidates, Final Releases and Patch Releases

Releases subsequent to the first release in a series usually involves a series of cherry-picks, the recipe is therefore slightly different.

## numba X.Y.Z

* [ ] Cherry-pick items from the X.Y.Z milestone into a cherry-pick PR.
* [ ] Update the "version support table" in the documentation with the final
  release date (FINAL ONLY) and add to cherry-pick PR
* [ ] Update `CHANGE_LOG` on cherry-pick PR
* [ ] Check if any dependency pinnings need an update (e.g. NumPy)
* [ ] Approve cherry-pick PR
* [ ] Merge cherry-pick PR to X.Y release branch.
  * [ ] https://github.com/numba/numba/pull/XXXX
* [ ] Review, merge and check execution of release notebook. (FINAL ONLY)
* [ ] Run the HEAD of the release branch through the build farm and confirm:
  * [ ] Build farm CPU testing has passed.
  * [ ] Build farm CUDA testing has passed
  * [ ] Build farm wheel testing has passed
* [ ] Annotated tag X.Y.Z on release branch (no `v` prefix).
  `git tag -am "Version X.Y.Z" X.Y.Z`
* [ ] Build and upload conda packages on buildfarm (check `upload`).
* [ ] Build wheels and sdist on the buildfarm (check "upload").
* [ ] Verify packages uploaded to Anaconda Cloud and move to
  `numba/label/main`.
* [ ] Upload wheels and sdist to PyPI (upload from `ci_artifacts`).
* [ ] Verify wheels for all platforms arrived on PyPi.
* [ ] Verify ReadTheDocs build.
* [ ] Create a release on Github at https://github.com/numba/numba/releases (FINAL ONLY).
* [ ] Post link to X and to Mastodon and...
* [ ] Post announcement to discourse group and ping the release testers group
  using `@RC_Testers` (RC ONLY).
* [ ] Post link to python-announce-list@python.org.

### Post release

* [ ] Cherry-pick change-log and version support table modifications to `main`
* [ ] Snapshot Build Farm config
* [ ] Clean up `ci_artifacts` by moving files to subdirectories
* [ ] Update release checklist template with any additional bullet points that
      may have arisen during the release.
* [ ] Ping Anaconda Distro team to trigger a build for `defaults` (FINAL ONLY).
* [ ] Close milestone (and then close this release issue).

Open a subsequent release checklist.

Generating Release Notes

The script maint/gitlog2changelog.py is used to generate release notes. To prepare to use it:

  • Install dependencies: conda install docopt pygithub gitpython.

  • Generate a fine-grained Personal Access Token on Github with read access to public repositories. This can be done in Github Personal Access Tokens settings.

  • Establish the base commit for the changelog. This is the common commit between main and the last release branch, which can be determined by running git merge-base main <branch>. For example, branch may be release0.58 if generating the release notes for the 0.59 release.

The script can then be invoked in the root of the repository with:

python maint/gitlog2changelog.py --token="<token>" --beginning="<base commit>" \
                                 --repo="numba/numba" --digits=4

This uses the token and commit established above. The --digits arguments specifies the number of digits in pull request numbers - this is presently 4 but will roll over to 5 soon after the time of writing - when that happens, it will be necessary to run the script twice, once with --digits=4 and once with --digits=5 and combine the results.

The script should output the release notes in a form that can be pasted into the top of the CHANGE_LOG file in the repo root. Truncated example output looks like:

Pull-Requests:

* PR `#8990 <https://github.com/numba/numba/pull/8990>`_: Removed extra block copying in InlineWorker (`kc611 <https://github.com/kc611>`_)
* PR `#9048 <https://github.com/numba/numba/pull/9048>`_: Dynamically allocate parfor schedule. (`DrTodd13 <https://github.com/DrTodd13>`_)

<... some output omitted ...>

Authors:

* `apmasell <https://github.com/apmasell>`_
* `DrTodd13 <https://github.com/DrTodd13>`_

<... some output omitted ...>