chore: reordering in Array.Basic

.github/ISSUE_TEMPLATE/bug_report.md

@@ -25,7 +25,7 @@ Please put an X between the brackets as you perform the following steps:
 
 ### Context
 
-[Broader context that the issue occurred in. If there was any prior discussion on [the Lean Zulip](https://leanprover.zulipchat.com), link it here as well.]
+[Broader context that the issue occured in. If there was any prior discussion on [the Lean Zulip](https://leanprover.zulipchat.com), link it here as well.]
 
 ### Steps to Reproduce
 
@@ -39,7 +39,7 @@ Please put an X between the brackets as you perform the following steps:
 
 ### Versions
 
-[Output of `#version` or `#eval Lean.versionString`]
+[Output of `#eval Lean.versionString`]
 [OS version, if not using live.lean-lang.org.]
 
 ### Additional Information

.github/PULL_REQUEST_TEMPLATE.md

@@ -5,10 +5,6 @@
 * Include the link to your `RFC` or `bug` issue in the description.
 * If the issue does not already have approval from a developer, submit the PR as draft.
 * The PR title/description will become the commit message. Keep it up-to-date as the PR evolves.
-* For `feat/fix` PRs, the first paragraph starting with "This PR" must be present and will become a
-  changelog entry unless the PR is labeled with `no-changelog`. If the PR does not have this label,
-  it must instead be categorized with one of the `changelog-*` labels (which will be done by a
-  reviewer for external PRs).
 * A toolchain of the form `leanprover/lean4-pr-releases:pr-release-NNNN` for Linux and M-series Macs will be generated upon build. To generate binaries for Windows and Intel-based Macs as well, write a comment containing `release-ci` on its own line.
 * If you rebase your PR onto `nightly-with-mathlib` then CI will test Mathlib against your PR.
 * You can manage the `awaiting-review`, `awaiting-author`, and `WIP` labels yourself, by writing a comment containing one of these labels on its own line.
@@ -16,6 +12,4 @@
 
 ---
 
-This PR <short changelog summary for feat/fix, see above>.
-
-Closes <`RFC` or `bug` issue number fixed by this PR, if any>
+Closes #0000 (`RFC` or `bug` issue number fixed by this PR, if any)

.github/actionlint.yaml

@@ -1,5 +0,0 @@
-self-hosted-runner:
-  labels:
-    - nscloud-ubuntu-22.04-amd64-4x16
-    - nscloud-ubuntu-22.04-amd64-8x16
-    - nscloud-macos-sonoma-arm64-6x14

.github/dependabot.yml

@@ -1,8 +0,0 @@
-version: 2
-updates:
-  - package-ecosystem: "github-actions"
-    directory: "/"
-    schedule:
-      interval: "monthly"
-    commit-message:
-      prefix: "chore: CI"

.github/workflows/actionlint.yml

@@ -17,6 +17,6 @@ jobs:
     - name: Checkout
       uses: actions/checkout@v4
     - name: actionlint
-      uses: raven-actions/actionlint@v2
+      uses: raven-actions/actionlint@v1
       with:
         pyflakes: false # we do not use python scripts

.github/workflows/awaiting-manual.yml

@@ -1,38 +0,0 @@
-name: Check awaiting-manual label
-
-on:
-  merge_group:
-  pull_request:
-    types: [opened, synchronize, reopened, labeled, unlabeled]
-
-jobs:
-  check-awaiting-manual:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Check awaiting-manual label
-        id: check-awaiting-manual-label
-        if: github.event_name == 'pull_request'
-        uses: actions/github-script@v7
-        with:
-          script: |
-            const { labels, number: prNumber } = context.payload.pull_request;
-            const hasAwaiting = labels.some(label => label.name == "awaiting-manual");
-            const hasBreaks = labels.some(label => label.name == "breaks-manual");
-            const hasBuilds = labels.some(label => label.name == "builds-manual");
-
-            if (hasAwaiting && hasBreaks) {
-              core.setFailed('PR has both "awaiting-manual" and "breaks-manual" labels.');
-            } else if (hasAwaiting && !hasBreaks && !hasBuilds) {
-              core.info('PR is marked "awaiting-manual" but neither "breaks-manual" nor "builds-manual" labels are present.');
-              core.setOutput('awaiting', 'true');
-            }
-      
-      - name: Wait for manual compatibility
-        if: github.event_name == 'pull_request' && steps.check-awaiting-manual-label.outputs.awaiting == 'true'
-        run: |
-          echo "::notice title=Awaiting manual::PR is marked 'awaiting-manual' but neither 'breaks-manual' nor 'builds-manual' labels are present."
-          echo "This check will remain in progress until the PR is updated with appropriate manual compatibility labels."
-          # Keep the job running indefinitely to show "in progress" status
-          while true; do
-            sleep 3600  # Sleep for 1 hour at a time
-          done

.github/workflows/awaiting-mathlib.yml

@@ -1,38 +0,0 @@
-name: Check awaiting-mathlib label
-
-on:
-  merge_group:
-  pull_request:
-    types: [opened, synchronize, reopened, labeled, unlabeled]
-
-jobs:
-  check-awaiting-mathlib:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Check awaiting-mathlib label
-        id: check-awaiting-mathlib-label
-        if: github.event_name == 'pull_request'
-        uses: actions/github-script@v7
-        with:
-          script: |
-            const { labels, number: prNumber } = context.payload.pull_request;
-            const hasAwaiting = labels.some(label => label.name == "awaiting-mathlib");
-            const hasBreaks = labels.some(label => label.name == "breaks-mathlib");
-            const hasBuilds = labels.some(label => label.name == "builds-mathlib");
-
-            if (hasAwaiting && hasBreaks) {
-              core.setFailed('PR has both "awaiting-mathlib" and "breaks-mathlib" labels.');
-            } else if (hasAwaiting && !hasBreaks && !hasBuilds) {
-              core.info('PR is marked "awaiting-mathlib" but neither "breaks-mathlib" nor "builds-mathlib" labels are present.');
-              core.setOutput('awaiting', 'true');
-            }
-      
-      - name: Wait for mathlib compatibility
-        if: github.event_name == 'pull_request' && steps.check-awaiting-mathlib-label.outputs.awaiting == 'true'
-        run: |
-          echo "::notice title=Awaiting mathlib::PR is marked 'awaiting-mathlib' but neither 'breaks-mathlib' nor 'builds-mathlib' labels are present."
-          echo "This check will remain in progress until the PR is updated with appropriate mathlib compatibility labels."
-          # Keep the job running indefinitely to show "in progress" status
-          while true; do
-            sleep 3600  # Sleep for 1 hour at a time
-          done

.github/workflows/build-template.yml

@@ -1,274 +0,0 @@
-# instantiated by ci.yml
-name: build-template
-on:
-  workflow_call:
-    inputs:
-      check-level:
-        type: string
-        required: true
-      config:
-        type: string
-        required: true
-      nightly:
-        type: string
-        required: true
-      LEAN_VERSION_MAJOR:
-        type: string
-        required: true
-      LEAN_VERSION_MINOR:
-        type: string
-        required: true
-      LEAN_VERSION_PATCH:
-        type: string
-        required: true
-      LEAN_SPECIAL_VERSION_DESC:
-        type: string
-        required: true
-      RELEASE_TAG:
-        type: string
-        required: true
-
-jobs:
-  build:
-    if: github.event_name != 'schedule' || github.repository == 'leanprover/lean4'
-    strategy:
-      matrix:
-        include: ${{fromJson(inputs.config)}}
-      # complete all jobs
-      fail-fast: false
-    runs-on: ${{ matrix.os }}
-    defaults:
-      run:
-        shell: ${{ matrix.shell || 'nix develop -c bash -euxo pipefail {0}' }}
-    name: ${{ matrix.name }}
-    env:
-      # must be inside workspace
-      CCACHE_DIR: ${{ github.workspace }}/.ccache
-      CCACHE_COMPRESS: true
-      # current cache limit
-      CCACHE_MAXSIZE: 400M
-      # squelch error message about missing nixpkgs channel
-      NIX_BUILD_SHELL: bash
-      LSAN_OPTIONS: max_leaks=10
-      # somehow MinGW clang64 (or cmake?) defaults to `g++` even though it doesn't exist
-      CXX: c++
-      MACOSX_DEPLOYMENT_TARGET: 10.15
-    steps:
-      - name: Install Nix
-        uses: DeterminateSystems/nix-installer-action@main
-        if: runner.os == 'Linux' && !matrix.cmultilib
-      - name: Install MSYS2
-        uses: msys2/setup-msys2@v2
-        with:
-          msystem: clang64
-          # `:` means do not prefix with msystem
-          pacboy: "make: python: cmake clang ccache gmp libuv git: zip: unzip: diffutils: binutils: tree: zstd tar:"
-        if: runner.os == 'Windows'
-      - name: Install Brew Packages
-        run: |
-          brew install ccache tree zstd coreutils gmp libuv
-        if: runner.os == 'macOS'
-      - name: Checkout
-        uses: actions/checkout@v4
-        with:
-          # the default is to use a virtual merge commit between the PR and master: just use the PR
-          ref: ${{ github.event.pull_request.head.sha }}
-      - name: Open Nix shell once
-        run: true
-        if: runner.os == 'Linux'
-      # Do check out some CI-relevant files from virtual merge commit to accommodate CI changes on
-      # master (as the workflow files themselves are always taken from the merge)
-      # (needs to be after "Install *" to use the right shell)
-      - name: CI Merge Checkout
-        run: |
-          git fetch --depth=1 origin ${{ github.sha }}
-          git checkout FETCH_HEAD flake.nix flake.lock script/prepare-* tests/lean/run/importStructure.lean
-        if: github.event_name == 'pull_request'
-      # (needs to be after "Checkout" so files don't get overridden)
-      - name: Setup emsdk
-        uses: mymindstorm/setup-emsdk@v14
-        with:
-          version: 3.1.44
-          actions-cache-folder: emsdk
-        if: matrix.wasm
-      - name: Install 32bit c libs
-        run: |
-          sudo dpkg --add-architecture i386
-          sudo apt-get update
-          sudo apt-get install -y gcc-multilib g++-multilib ccache libuv1-dev:i386 pkgconf:i386
-        if: matrix.cmultilib
-      - name: Restore Cache
-        id: restore-cache
-        uses: actions/cache/restore@v4
-        with:
-          # NOTE: must be in sync with `save` below and with `restore-cache` in `update-stage0.yml`
-          path: |
-            .ccache
-            ${{ matrix.name == 'Linux Lake (cached)' && 'build/stage1/**/*.trace
-            build/stage1/**/*.olean*
-            build/stage1/**/*.ilean
-            build/stage1/**/*.ir
-            build/stage1/**/*.c
-            build/stage1/**/*.c.o*' || '' }}
-          key: ${{ matrix.name }}-build-v3-${{ github.sha }}
-          # fall back to (latest) previous cache
-          restore-keys: |
-            ${{ matrix.name }}-build-v3
-      # open nix-shell once for initial setup
-      - name: Setup
-        run: |
-          ccache --zero-stats
-        if: runner.os == 'Linux'
-      - name: Set up env
-        run: |
-          echo "NPROC=$(nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 4)" >> $GITHUB_ENV
-          if ! diff src/stdlib_flags.h stage0/src/stdlib_flags.h; then
-            echo "src/stdlib_flags.h and stage0/src/stdlib_flags.h differ, will test and pack stage 2"
-            echo "TARGET_STAGE=stage2" >> $GITHUB_ENV
-          else
-            echo "TARGET_STAGE=stage1" >> $GITHUB_ENV
-          fi
-      - name: Build
-        run: |
-          ulimit -c unlimited  # coredumps
-          [ -d build ] || mkdir build
-          cd build
-          # arguments passed to `cmake`
-          OPTIONS=(-DLEAN_EXTRA_MAKE_OPTS=-DwarningAsError=true)
-          if [[ -n '${{ matrix.release }}' ]]; then
-            # this also enables githash embedding into stage 1 library, which prohibits reusing
-            # `.olean`s across commits, so we don't do it in the fast non-release CI
-            OPTIONS+=(-DCHECK_OLEAN_VERSION=ON)
-          fi
-          if [[ -n '${{ matrix.cross_target }}' ]]; then
-            # used by `prepare-llvm`
-            export EXTRA_FLAGS=--target=${{ matrix.cross_target }}
-            OPTIONS+=(-DLEAN_PLATFORM_TARGET=${{ matrix.cross_target }})
-          fi
-          if [[ -n '${{ matrix.prepare-llvm }}' ]]; then
-            wget -q ${{ matrix.llvm-url }}
-            PREPARE="$(${{ matrix.prepare-llvm }})"
-            if [ "$TARGET_STAGE" == "stage2" ]; then
-              cp -r stage1 stage2
-            fi
-            eval "OPTIONS+=($PREPARE)"
-          fi
-          if [[ -n '${{ matrix.release }}' && -n '${{ inputs.nightly }}' ]]; then
-            OPTIONS+=(-DLEAN_SPECIAL_VERSION_DESC=${{ inputs.nightly }})
-          fi
-          if [[ -n '${{ matrix.release }}' && -n '${{ inputs.RELEASE_TAG }}' ]]; then
-            OPTIONS+=(-DLEAN_VERSION_MAJOR=${{ inputs.LEAN_VERSION_MAJOR }})
-            OPTIONS+=(-DLEAN_VERSION_MINOR=${{ inputs.LEAN_VERSION_MINOR }})
-            OPTIONS+=(-DLEAN_VERSION_PATCH=${{ inputs.LEAN_VERSION_PATCH }})
-            OPTIONS+=(-DLEAN_VERSION_IS_RELEASE=1)
-            OPTIONS+=(-DLEAN_SPECIAL_VERSION_DESC=${{ inputs.LEAN_SPECIAL_VERSION_DESC }})
-          fi
-          # contortion to support empty OPTIONS with old macOS bash
-          cmake .. --preset ${{ matrix.CMAKE_PRESET || 'release' }} -B . ${{ matrix.CMAKE_OPTIONS }} ${OPTIONS[@]+"${OPTIONS[@]}"} -DLEAN_INSTALL_PREFIX=$PWD/..
-          time make $TARGET_STAGE -j$NPROC
-      - name: Install
-        run: |
-          make -C build/$TARGET_STAGE install
-      - name: Check Binaries
-        run: ${{ matrix.binary-check }} lean-*/bin/* || true
-      - name: Count binary symbols
-        run: |
-          for f in lean-*/bin/*; do
-            echo "$f: $(nm $f | grep " T " | wc -l) exported symbols"
-          done
-        if: matrix.name == 'Windows'
-      - name: List Install Tree
-        run: |
-          # omit contents of Init/, ...
-          tree --du -h lean-*-* | grep -E ' (Init|Lean|Lake|LICENSE|[a-z])'
-      - name: Pack
-        run: |
-          dir=$(echo lean-*-*)
-          mkdir pack
-          # high-compression tar.zst + zip for release, fast tar.zst otherwise
-          if [[ '${{ startsWith(github.ref, 'refs/tags/') && matrix.release }}' == true || -n '${{ inputs.nightly }}' || -n '${{ inputs.RELEASE_TAG }}' ]]; then
-            ${{ matrix.tar || 'tar' }} cf - $dir | zstd -T0 --no-progress -19 -o pack/$dir.tar.zst
-            zip -rq pack/$dir.zip $dir
-          else
-            ${{ matrix.tar || 'tar' }} cf - $dir | zstd -T0 --no-progress -o pack/$dir.tar.zst
-          fi
-      - uses: actions/upload-artifact@v4
-        if: matrix.release
-        with:
-          name: build-${{ matrix.name }}
-          path: pack/*
-      - name: Lean stats
-        run: |
-          build/stage1/bin/lean --stats src/Lean.lean -Dexperimental.module=true
-        if: ${{ !matrix.cross }}
-      - name: Test
-        id: test
-        run: |
-          ulimit -c unlimited  # coredumps
-          time ctest --preset ${{ matrix.CMAKE_PRESET || 'release' }} --test-dir build/$TARGET_STAGE -j$NPROC --output-junit test-results.xml
-        if: (matrix.wasm || !matrix.cross) && (inputs.check-level >= 1 || matrix.test)
-      - name: Test Summary
-        uses: test-summary/action@v2
-        with:
-          paths: build/${{ env.TARGET_STAGE }}/test-results.xml
-        # prefix `if` above with `always` so it's run even if tests failed
-        if: always() && steps.test.conclusion != 'skipped'
-      - name: Check Test Binary
-        run: ${{ matrix.binary-check }} tests/compiler/534.lean.out
-        if: (!matrix.cross) && steps.test.conclusion != 'skipped'
-      - name: Build Stage 2
-        run: |
-          make -C build -j$NPROC stage2
-        if: matrix.test-speedcenter
-      - name: Check Stage 3
-        run: |
-          make -C build -j$NPROC check-stage3
-        if: matrix.check-stage3
-      - name: Test Speedcenter Benchmarks
-        run: |
-          # Necessary for some timing metrics but does not work on Namespace runners
-          # and we just want to test that the benchmarks run at all here
-          #echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid
-          export BUILD=$PWD/build PATH=$PWD/build/stage1/bin:$PATH
-          cd tests/bench
-          nix shell .#temci -c temci exec --config speedcenter.yaml --included_blocks fast --runs 1
-        if: matrix.test-speedcenter
-      - name: Check rebootstrap
-        run: |
-          set -e
-          # clean rebuild in case of Makefile changes/Lake does not detect uncommited stage 0
-          # changes yet
-          make -C build update-stage0
-          make -C build/stage1 clean-stdlib
-          time make -C build -j$NPROC
-          time ctest --preset ${{ matrix.CMAKE_PRESET || 'release' }} --test-dir build/stage1 -j$NPROC
-        if: matrix.check-rebootstrap
-      - name: CCache stats
-        if: always()
-        run: ccache -s
-      - name: Show stacktrace for coredumps
-        if: failure() && runner.os == 'Linux'
-        run: |
-          for c in $(find . -name core); do
-            progbin="$(file $c | sed "s/.*execfn: '\([^']*\)'.*/\1/")"
-            echo bt | $GDB/bin/gdb -q $progbin $c || true
-          done
-      - name: Save Cache
-        if: always() && steps.restore-cache.outputs.cache-hit != 'true'
-        uses: actions/cache/save@v4
-        with:
-          # NOTE: must be in sync with `restore` above
-          path: |
-            .ccache
-            ${{ matrix.name == 'Linux Lake (cached)' && 'build/stage1/**/*.trace
-            build/stage1/**/*.olean*
-            build/stage1/**/*.ilean
-            build/stage1/**/*.ir
-            build/stage1/**/*.c
-            build/stage1/**/*.c.o*' || '' }}
-          key: ${{ steps.restore-cache.outputs.cache-primary-key }}
-      - name: Upload Build Artifact
-        if: always() && matrix.name == 'Linux Lake (cached)'
-        uses: actions/upload-artifact@v4
-        with:
-          path: build

.github/workflows/check-prelude.yml

@@ -11,10 +11,7 @@ jobs:
         with:
           # the default is to use a virtual merge commit between the PR and master: just use the PR
           ref: ${{ github.event.pull_request.head.sha }}
-          sparse-checkout: |
-            src/Lean
-            src/Std
-            src/lake/Lake
+          sparse-checkout: src/Lean
       - name: Check Prelude
         run: |
           failed_files=""
@@ -22,8 +19,8 @@ jobs:
             if ! grep -q "^prelude$" "$file"; then
               failed_files="$failed_files$file\n"
             fi
-          done < <(find src/Lean src/Std src/lake/Lake -name '*.lean' -print0)
+          done < <(find src/Lean -name '*.lean' -print0)
           if [ -n "$failed_files" ]; then
             echo -e "The following files should use 'prelude':\n$failed_files"
             exit 1
-          fi
+          fi

.github/workflows/check-stage0.yml

@@ -20,7 +20,9 @@ jobs:
 
     - name: Identify stage0 changes
       run: |
-        git diff "${BASE:-HEAD^}..HEAD" --name-only -- stage0/stdlib > "$RUNNER_TEMP/stage0" || true
+        git diff "${BASE:-HEAD^}..HEAD" --name-only -- stage0 |
+          grep -v -x -F $'stage0/src/stdlib_flags.h\nstage0/src/lean.mk.in' \
+          > "$RUNNER_TEMP/stage0" || true
         if test -s "$RUNNER_TEMP/stage0"
         then
           echo "CHANGES=yes" >> "$GITHUB_ENV"

.github/workflows/ci.yml

@@ -36,9 +36,7 @@ jobs:
       # 2: PRs with `release-ci` label, releases (incl. nightlies)
       check-level: ${{ steps.set-level.outputs.check-level }}
       # The build matrix, dynamically generated here
-      matrix: ${{ steps.set-matrix.outputs.matrix }}
-      # secondary build jobs that should not block the CI success/merge queue
-      matrix-secondary: ${{ steps.set-matrix.outputs.matrix-secondary }}
+      matrix: ${{ steps.set-matrix.outputs.result }}
       # Should we make a nightly release? If so, this output contains the lean version string, else it is empty
       nightly: ${{ steps.set-nightly.outputs.nightly }}
       # Should this be the CI for a tagged release?
@@ -103,13 +101,6 @@ jobs:
             echo "Tag ${TAG_NAME} did not match SemVer regex."
           fi
 
-      - name: Check for custom releases (e.g., not in the main lean repository)
-        if: startsWith(github.ref, 'refs/tags/') && github.repository != 'leanprover/lean4'
-        id: set-release-custom
-        run: |
-          TAG_NAME="${GITHUB_REF##*/}"
-          echo "RELEASE_TAG=$TAG_NAME" >> "$GITHUB_OUTPUT"
-
       - name: Set check level
         id: set-level
         # We do not use github.event.pull_request.labels.*.name here because
@@ -118,7 +109,7 @@ jobs:
         run: |
           check_level=0
 
-          if [[ -n "${{ steps.set-nightly.outputs.nightly }}" || -n "${{ steps.set-release.outputs.RELEASE_TAG }}" || -n "${{ steps.set-release-custom.outputs.RELEASE_TAG }}" ]]; then
+          if [[ -n "${{ steps.set-nightly.outputs.nightly }}" || -n "${{ steps.set-release.outputs.RELEASE_TAG }}" ]]; then
             check_level=2
           elif [[ "${{ github.event_name }}" != "pull_request" ]]; then
             check_level=1
@@ -144,70 +135,48 @@ jobs:
             console.log(`level: ${level}`);
             // use large runners where available (original repo)
             let large = ${{ github.repository == 'leanprover/lean4' }};
-            const isPr = "${{ github.event_name }}" == "pull_request";
-            const isPushToMaster = "${{ github.event_name }}" == "push" && "${{ github.ref_name }}" == "master";
             let matrix = [
-              /* TODO: to be updated to new LLVM
               {
+                // portable release build: use channel with older glibc (2.27)
                 "name": "Linux LLVM",
                 "os": "ubuntu-latest",
                 "release": false,
                 "check-level": 2,
                 "shell": "nix develop .#oldGlibc -c bash -euxo pipefail {0}",
-                "llvm-url": "https://github.com/leanprover/lean-llvm/releases/download/19.1.2/lean-llvm-x86_64-linux-gnu.tar.zst",
+                "llvm-url": "https://github.com/leanprover/lean-llvm/releases/download/15.0.1/lean-llvm-x86_64-linux-gnu.tar.zst",
                 "prepare-llvm": "../script/prepare-llvm-linux.sh lean-llvm*",
                 "binary-check": "ldd -v",
                 // foreign code may be linked against more recent glibc
                 // reverse-ffi needs to be updated to link to LLVM libraries
                 "CTEST_OPTIONS": "-E 'foreign|leanlaketest_reverse-ffi'",
                 "CMAKE_OPTIONS": "-DLLVM=ON -DLLVM_CONFIG=${GITHUB_WORKSPACE}/build/llvm-host/bin/llvm-config"
-              }, */
+              },
               {
-                // portable release build: use channel with older glibc (2.26)
                 "name": "Linux release",
-                "os": "ubuntu-latest",
+                "os": large ? "nscloud-ubuntu-22.04-amd64-4x8" : "ubuntu-latest",
                 "release": true,
-                // Special handling for release jobs. We want:
-                // 1. To run it in PRs so developers get PR toolchains (so secondary is sufficient)
-                // 2. To skip it in merge queues as it takes longer than the
-                //    Linux lake build and adds little value in the merge queue
-                // 3. To run it in release (obviously)
-                // 4. To run it for pushes to master so that pushes to master have a Linux toolchain
-                //    available as an artifact for Grove to use.
-                "check-level": (isPr || isPushToMaster) ? 0 : 2,
-                "secondary": isPr,
+                "check-level": 0,
                 "shell": "nix develop .#oldGlibc -c bash -euxo pipefail {0}",
-                "llvm-url": "https://github.com/leanprover/lean-llvm/releases/download/19.1.2/lean-llvm-x86_64-linux-gnu.tar.zst",
+                "llvm-url": "https://github.com/leanprover/lean-llvm/releases/download/15.0.1/lean-llvm-x86_64-linux-gnu.tar.zst",
                 "prepare-llvm": "../script/prepare-llvm-linux.sh lean-llvm*",
                 "binary-check": "ldd -v",
                 // foreign code may be linked against more recent glibc
                 "CTEST_OPTIONS": "-E 'foreign'"
               },
               {
-                "name": "Linux Lake",
-                "os": large ? "nscloud-ubuntu-22.04-amd64-8x16" : "ubuntu-latest",
-                "check-level": 0,
-                "test": true,
-                "check-rebootstrap": level >= 1,
+                "name": "Linux",
+                "os": large ? "nscloud-ubuntu-22.04-amd64-4x8" : "ubuntu-latest",
                 "check-stage3": level >= 2,
-                // NOTE: `test-speedcenter` currently seems to be broken on `ubuntu-latest`
-                "test-speedcenter": large && level >= 2,
-                "CMAKE_OPTIONS": "-DUSE_LAKE=ON",
+                "test-speedcenter": level >= 2,
+                "check-level": 1,
               },
               {
-                "name": "Linux Lake (cached)",
-                "os": "ubuntu-latest",
-                "check-level": (isPr || isPushToMaster) ? 0 : 2,
-                "secondary": true,
-                "CMAKE_OPTIONS": "-DUSE_LAKE=ON",
-              },
-              {
-                "name": "Linux Reldebug",
+                "name": "Linux Debug",
                 "os": "ubuntu-latest",
                 "check-level": 2,
-                "CMAKE_PRESET": "reldebug",
-                // exclude seriously slow/stackoverflowing tests
-                "CTEST_OPTIONS": "-E 'interactivetest|leanpkgtest|laketest|benchtest|bv_bitblast_stress|3807'"
+                "CMAKE_PRESET": "debug",
+                // exclude seriously slow tests
+                "CTEST_OPTIONS": "-E 'interactivetest|leanpkgtest|laketest|benchtest|bv_bitblast_stress'"
               },
               // TODO: suddenly started failing in CI
               /*{
@@ -225,114 +194,265 @@ jobs:
                 "release": true,
                 "check-level": 2,
                 "shell": "bash -euxo pipefail {0}",
-                "llvm-url": "https://github.com/leanprover/lean-llvm/releases/download/19.1.2/lean-llvm-x86_64-apple-darwin.tar.zst",
+                "llvm-url": "https://github.com/leanprover/lean-llvm/releases/download/15.0.1/lean-llvm-x86_64-apple-darwin.tar.zst",
                 "prepare-llvm": "../script/prepare-llvm-macos.sh lean-llvm*",
                 "binary-check": "otool -L",
                 "tar": "gtar" // https://github.com/actions/runner-images/issues/2619
               },
               {
                 "name": "macOS aarch64",
-                // standard GH runner only comes with 7GB so use large runner if possible when running tests
-                "os": large && !isPr ? "nscloud-macos-sonoma-arm64-6x14" : "macos-14",
+                "os": "macos-14",
                 "CMAKE_OPTIONS": "-DLEAN_INSTALL_SUFFIX=-darwin_aarch64",
                 "release": true,
+                "check-level": 0,
                 "shell": "bash -euxo pipefail {0}",
-                "llvm-url": "https://github.com/leanprover/lean-llvm/releases/download/19.1.2/lean-llvm-aarch64-apple-darwin.tar.zst",
+                "llvm-url": "https://github.com/leanprover/lean-llvm/releases/download/15.0.1/lean-llvm-aarch64-apple-darwin.tar.zst",
                 "prepare-llvm": "../script/prepare-llvm-macos.sh lean-llvm*",
                 "binary-check": "otool -L",
-                "tar": "gtar", // https://github.com/actions/runner-images/issues/2619
-                // See "Linux release" for release job levels; Grove is not a concern here
-                "check-level": isPr ? 0 : 2,
-                "secondary": isPr,
+                "tar": "gtar" // https://github.com/actions/runner-images/issues/2619
               },
               {
                 "name": "Windows",
-                "os": large && level == 2 ? "namespace-profile-windows-amd64-4x16" : "windows-2022",
+                "os": "windows-2022",
                 "release": true,
                 "check-level": 2,
                 "shell": "msys2 {0}",
-                "CMAKE_OPTIONS": "-G \"Unix Makefiles\"",
+                "CMAKE_OPTIONS": "-G \"Unix Makefiles\" -DUSE_GMP=OFF",
                 // for reasons unknown, interactivetests are flaky on Windows
                 "CTEST_OPTIONS": "--repeat until-pass:2",
-                "llvm-url": "https://github.com/leanprover/lean-llvm/releases/download/19.1.2/lean-llvm-x86_64-w64-windows-gnu.tar.zst",
+                "llvm-url": "https://github.com/leanprover/lean-llvm/releases/download/15.0.1/lean-llvm-x86_64-w64-windows-gnu.tar.zst",
                 "prepare-llvm": "../script/prepare-llvm-mingw.sh lean-llvm*",
                 "binary-check": "ldd"
               },
               {
                 "name": "Linux aarch64",
-                "os": "nscloud-ubuntu-22.04-arm64-4x16",
-                "CMAKE_OPTIONS": "-DLEAN_INSTALL_SUFFIX=-linux_aarch64",
+                "os": "nscloud-ubuntu-22.04-arm64-4x8",
+                "CMAKE_OPTIONS": "-DUSE_GMP=OFF -DLEAN_INSTALL_SUFFIX=-linux_aarch64",
                 "release": true,
                 "check-level": 2,
                 "shell": "nix develop .#oldGlibcAArch -c bash -euxo pipefail {0}",
-                "llvm-url": "https://github.com/leanprover/lean-llvm/releases/download/19.1.2/lean-llvm-aarch64-linux-gnu.tar.zst",
+                "llvm-url": "https://github.com/leanprover/lean-llvm/releases/download/15.0.1/lean-llvm-aarch64-linux-gnu.tar.zst",
                 "prepare-llvm": "../script/prepare-llvm-linux.sh lean-llvm*"
               },
-              // Started running out of memory building expensive modules, a 2GB heap is just not that much even before fragmentation
-              //{
-              //  "name": "Linux 32bit",
-              //  "os": "ubuntu-latest",
-              //  // Use 32bit on stage0 and stage1 to keep oleans compatible
-              //  "CMAKE_OPTIONS": "-DSTAGE0_USE_GMP=OFF -DSTAGE0_LEAN_EXTRA_CXX_FLAGS='-m32' -DSTAGE0_LEANC_OPTS='-m32' -DSTAGE0_MMAP=OFF -DUSE_GMP=OFF -DLEAN_EXTRA_CXX_FLAGS='-m32' -DLEANC_OPTS='-m32' -DMMAP=OFF -DLEAN_INSTALL_SUFFIX=-linux_x86 -DCMAKE_LIBRARY_PATH=/usr/lib/i386-linux-gnu/ -DSTAGE0_CMAKE_LIBRARY_PATH=/usr/lib/i386-linux-gnu/ -DPKG_CONFIG_EXECUTABLE=/usr/bin/i386-linux-gnu-pkg-config",
-              //  "cmultilib": true,
-              //  "release": true,
-              //  "check-level": 2,
-              //  "cross": true,
-              //  "shell": "bash -euxo pipefail {0}"
-              //}
-              // {
-              //   "name": "Web Assembly",
-              //   "os": "ubuntu-latest",
-              //   // Build a native 32bit binary in stage0 and use it to compile the oleans and the wasm build
-              //   "CMAKE_OPTIONS": "-DCMAKE_C_COMPILER_WORKS=1 -DSTAGE0_USE_GMP=OFF -DSTAGE0_LEAN_EXTRA_CXX_FLAGS='-m32' -DSTAGE0_LEANC_OPTS='-m32' -DSTAGE0_CMAKE_CXX_COMPILER=clang++ -DSTAGE0_CMAKE_C_COMPILER=clang -DSTAGE0_CMAKE_EXECUTABLE_SUFFIX=\"\" -DUSE_GMP=OFF -DMMAP=OFF -DSTAGE0_MMAP=OFF -DCMAKE_AR=../emsdk/emsdk-main/upstream/emscripten/emar -DCMAKE_TOOLCHAIN_FILE=../emsdk/emsdk-main/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake -DLEAN_INSTALL_SUFFIX=-linux_wasm32 -DSTAGE0_CMAKE_LIBRARY_PATH=/usr/lib/i386-linux-gnu/",
-              //   "wasm": true,
-              //   "cmultilib": true,
-              //   "release": true,
-              //   "check-level": 2,
-              //   "cross": true,
-              //   "shell": "bash -euxo pipefail {0}",
-              //   // Just a few selected tests because wasm is slow
-              //   "CTEST_OPTIONS": "-R \"leantest_1007\\.lean|leantest_Format\\.lean|leanruntest\\_1037.lean|leanruntest_ac_rfl\\.lean|leanruntest_tempfile.lean\\.|leanruntest_libuv\\.lean\""
-              // }
+              {
+                "name": "Linux 32bit",
+                "os": "ubuntu-latest",
+                // Use 32bit on stage0 and stage1 to keep oleans compatible
+                "CMAKE_OPTIONS": "-DSTAGE0_USE_GMP=OFF -DSTAGE0_LEAN_EXTRA_CXX_FLAGS='-m32' -DSTAGE0_LEANC_OPTS='-m32' -DSTAGE0_MMAP=OFF -DUSE_GMP=OFF -DLEAN_EXTRA_CXX_FLAGS='-m32' -DLEANC_OPTS='-m32' -DMMAP=OFF -DLEAN_INSTALL_SUFFIX=-linux_x86 -DCMAKE_LIBRARY_PATH=/usr/lib/i386-linux-gnu/ -DSTAGE0_CMAKE_LIBRARY_PATH=/usr/lib/i386-linux-gnu/",
+                "cmultilib": true,
+                "release": true,
+                "check-level": 2,
+                "cross": true,
+                "shell": "bash -euxo pipefail {0}"
+              },
+              {
+                "name": "Web Assembly",
+                "os": "ubuntu-latest",
+                // Build a native 32bit binary in stage0 and use it to compile the oleans and the wasm build
+                "CMAKE_OPTIONS": "-DCMAKE_C_COMPILER_WORKS=1 -DSTAGE0_USE_GMP=OFF -DSTAGE0_LEAN_EXTRA_CXX_FLAGS='-m32' -DSTAGE0_LEANC_OPTS='-m32' -DSTAGE0_CMAKE_CXX_COMPILER=clang++ -DSTAGE0_CMAKE_C_COMPILER=clang -DSTAGE0_CMAKE_EXECUTABLE_SUFFIX=\"\" -DUSE_GMP=OFF -DMMAP=OFF -DSTAGE0_MMAP=OFF -DCMAKE_AR=../emsdk/emsdk-main/upstream/emscripten/emar -DCMAKE_TOOLCHAIN_FILE=../emsdk/emsdk-main/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake -DLEAN_INSTALL_SUFFIX=-linux_wasm32 -DSTAGE0_CMAKE_LIBRARY_PATH=/usr/lib/i386-linux-gnu/",
+                "wasm": true,
+                "cmultilib": true,
+                "release": true,
+                "check-level": 2,
+                "cross": true,
+                "shell": "bash -euxo pipefail {0}",
+                // Just a few selected tests because wasm is slow
+                "CTEST_OPTIONS": "-R \"leantest_1007\\.lean|leantest_Format\\.lean|leanruntest\\_1037.lean|leanruntest_ac_rfl\\.lean|leanruntest_libuv\\.lean\""
+              }
             ];
-            console.log(`matrix:\n${JSON.stringify(matrix, null, 2)}`);
-            matrix = matrix.filter((job) => level >= job["check-level"]);
-            core.setOutput('matrix', matrix.filter((job) => !job["secondary"]));
-            core.setOutput('matrix-secondary', matrix.filter((job) => job["secondary"]));
+            console.log(`matrix:\n${JSON.stringify(matrix, null, 2)}`)
+            return matrix.filter((job) => level >= job["check-level"])
 
   build:
+    needs: [configure]
     if: github.event_name != 'schedule' || github.repository == 'leanprover/lean4'
-    needs: [configure]
-    uses: ./.github/workflows/build-template.yml
-    with:
-      config: ${{needs.configure.outputs.matrix}}
-      check-level: ${{ needs.configure.outputs.check-level }}
-      nightly: ${{ needs.configure.outputs.nightly }}
-      LEAN_VERSION_MAJOR: ${{ needs.configure.outputs.LEAN_VERSION_MAJOR }}
-      LEAN_VERSION_MINOR: ${{ needs.configure.outputs.LEAN_VERSION_MINOR }}
-      LEAN_VERSION_PATCH: ${{ needs.configure.outputs.LEAN_VERSION_PATCH }}
-      LEAN_SPECIAL_VERSION_DESC: ${{ needs.configure.outputs.LEAN_SPECIAL_VERSION_DESC }}
-      RELEASE_TAG: ${{ needs.configure.outputs.RELEASE_TAG }}
-    secrets: inherit
-
-  # build jobs that should not be considered by `all-done` below
-  build-secondary:
-    needs: [configure]
-    if: needs.configure.outputs.matrix-secondary != '[]'
-    uses: ./.github/workflows/build-template.yml
-    with:
-      config: ${{needs.configure.outputs.matrix-secondary}}
-      check-level: ${{ needs.configure.outputs.check-level }}
-      nightly: ${{ needs.configure.outputs.nightly }}
-      LEAN_VERSION_MAJOR: ${{ needs.configure.outputs.LEAN_VERSION_MAJOR }}
-      LEAN_VERSION_MINOR: ${{ needs.configure.outputs.LEAN_VERSION_MINOR }}
-      LEAN_VERSION_PATCH: ${{ needs.configure.outputs.LEAN_VERSION_PATCH }}
-      LEAN_SPECIAL_VERSION_DESC: ${{ needs.configure.outputs.LEAN_SPECIAL_VERSION_DESC }}
-      RELEASE_TAG: ${{ needs.configure.outputs.RELEASE_TAG }}
-    secrets: inherit
+    strategy:
+      matrix:
+        include: ${{fromJson(needs.configure.outputs.matrix)}}
+      # complete all jobs
+      fail-fast: false
+    runs-on: ${{ matrix.os }}
+    defaults:
+      run:
+        shell: ${{ matrix.shell || 'nix develop -c bash -euxo pipefail {0}' }}
+    name: ${{ matrix.name }}
+    env:
+      # must be inside workspace
+      CCACHE_DIR: ${{ github.workspace }}/.ccache
+      CCACHE_COMPRESS: true
+      # current cache limit
+      CCACHE_MAXSIZE: 200M
+      # squelch error message about missing nixpkgs channel
+      NIX_BUILD_SHELL: bash
+      LSAN_OPTIONS: max_leaks=10
+      # somehow MinGW clang64 (or cmake?) defaults to `g++` even though it doesn't exist
+      CXX: c++
+      MACOSX_DEPLOYMENT_TARGET: 10.15
+    steps:
+      - name: Install Nix
+        uses: DeterminateSystems/nix-installer-action@main
+        if: runner.os == 'Linux' && !matrix.cmultilib
+      - name: Install MSYS2
+        uses: msys2/setup-msys2@v2
+        with:
+          msystem: clang64
+          # `:` means do not prefix with msystem
+          pacboy: "make: python: cmake clang ccache gmp libuv git: zip: unzip: diffutils: binutils: tree: zstd tar:"
+        if: runner.os == 'Windows'
+      - name: Install Brew Packages
+        run: |
+          brew install ccache tree zstd coreutils gmp libuv
+        if: runner.os == 'macOS'
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          # the default is to use a virtual merge commit between the PR and master: just use the PR
+          ref: ${{ github.event.pull_request.head.sha }}
+      # Do check out some CI-relevant files from virtual merge commit to accommodate CI changes on
+      # master (as the workflow files themselves are always taken from the merge)
+      # (needs to be after "Install *" to use the right shell)
+      - name: CI Merge Checkout
+        run: |
+          git fetch --depth=1 origin ${{ github.sha }}
+          git checkout FETCH_HEAD flake.nix flake.lock
+        if: github.event_name == 'pull_request'
+      # (needs to be after "Checkout" so files don't get overriden)
+      - name: Setup emsdk
+        uses: mymindstorm/setup-emsdk@v12
+        with:
+          version: 3.1.44
+          actions-cache-folder: emsdk
+        if: matrix.wasm
+      - name: Install 32bit c libs
+        run: |
+          sudo dpkg --add-architecture i386
+          sudo apt-get update
+          sudo apt-get install -y gcc-multilib g++-multilib ccache libuv1-dev:i386
+        if: matrix.cmultilib
+      - name: Cache
+        uses: actions/cache@v4
+        with:
+          path: .ccache
+          key: ${{ matrix.name }}-build-v3-${{ github.event.pull_request.head.sha }}
+          # fall back to (latest) previous cache
+          restore-keys: |
+            ${{ matrix.name }}-build-v3
+          save-always: true
+      # open nix-shell once for initial setup
+      - name: Setup
+        run: |
+          ccache --zero-stats
+        if: runner.os == 'Linux'
+      - name: Set up NPROC
+        run: |
+          echo "NPROC=$(nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 4)" >> $GITHUB_ENV
+      - name: Build
+        run: |
+          mkdir build
+          cd build
+          # arguments passed to `cmake`
+          # this also enables githash embedding into stage 1 library
+          OPTIONS=(-DCHECK_OLEAN_VERSION=ON)
+          OPTIONS+=(-DLEAN_EXTRA_MAKE_OPTS=-DwarningAsError=true)
+          if [[ -n '${{ matrix.cross_target }}' ]]; then
+            # used by `prepare-llvm`
+            export EXTRA_FLAGS=--target=${{ matrix.cross_target }}
+            OPTIONS+=(-DLEAN_PLATFORM_TARGET=${{ matrix.cross_target }})
+          fi
+          if [[ -n '${{ matrix.prepare-llvm }}' ]]; then
+            wget -q ${{ matrix.llvm-url }}
+            PREPARE="$(${{ matrix.prepare-llvm }})"
+            eval "OPTIONS+=($PREPARE)"
+          fi
+          if [[ -n '${{ matrix.release }}' && -n '${{ needs.configure.outputs.nightly }}' ]]; then
+            OPTIONS+=(-DLEAN_SPECIAL_VERSION_DESC=${{ needs.configure.outputs.nightly }})
+          fi
+          if [[ -n '${{ matrix.release }}' && -n '${{ needs.configure.outputs.RELEASE_TAG }}' ]]; then
+            OPTIONS+=(-DLEAN_VERSION_MAJOR=${{ needs.configure.outputs.LEAN_VERSION_MAJOR }})
+            OPTIONS+=(-DLEAN_VERSION_MINOR=${{ needs.configure.outputs.LEAN_VERSION_MINOR }})
+            OPTIONS+=(-DLEAN_VERSION_PATCH=${{ needs.configure.outputs.LEAN_VERSION_PATCH }})
+            OPTIONS+=(-DLEAN_VERSION_IS_RELEASE=1)
+            OPTIONS+=(-DLEAN_SPECIAL_VERSION_DESC=${{ needs.configure.outputs.LEAN_SPECIAL_VERSION_DESC }})
+          fi
+          # contortion to support empty OPTIONS with old macOS bash
+          cmake .. --preset ${{ matrix.CMAKE_PRESET || 'release' }} -B . ${{ matrix.CMAKE_OPTIONS }} ${OPTIONS[@]+"${OPTIONS[@]}"} -DLEAN_INSTALL_PREFIX=$PWD/..
+          time make -j$NPROC
+      - name: Install
+        run: |
+          make -C build install
+      - name: Check Binaries
+        run: ${{ matrix.binary-check }} lean-*/bin/* || true
+      - name: Count binary symbols
+        run: |
+          for f in lean-*/bin/*; do
+            echo "$f: $(nm $f | grep " T " | wc -l) exported symbols"
+          done
+        if: matrix.name == 'Windows'
+      - name: List Install Tree
+        run: |
+          # omit contents of Init/, ...
+          tree --du -h lean-*-* | grep -E ' (Init|Lean|Lake|LICENSE|[a-z])'
+      - name: Pack
+        run: |
+          dir=$(echo lean-*-*)
+          mkdir pack
+          # high-compression tar.zst + zip for release, fast tar.zst otherwise
+          if [[ '${{ startsWith(github.ref, 'refs/tags/') && matrix.release }}' == true || -n '${{ needs.configure.outputs.nightly }}' || -n '${{ needs.configure.outputs.RELEASE_TAG }}' ]]; then
+            ${{ matrix.tar || 'tar' }} cf - $dir | zstd -T0 --no-progress -19 -o pack/$dir.tar.zst
+            zip -rq pack/$dir.zip $dir
+          else
+            ${{ matrix.tar || 'tar' }} cf - $dir | zstd -T0 --no-progress -o pack/$dir.tar.zst
+          fi
+      - uses: actions/upload-artifact@v4
+        if: matrix.release
+        with:
+          name: build-${{ matrix.name }}
+          path: pack/*
+      - name: Lean stats
+        run: |
+          build/stage1/bin/lean --stats src/Lean.lean
+        if: ${{ !matrix.cross }}
+      - name: Test
+        id: test
+        run: |
+          time ctest --preset ${{ matrix.CMAKE_PRESET || 'release' }} --test-dir build/stage1 -j$NPROC --output-junit test-results.xml ${{ matrix.CTEST_OPTIONS }}
+        if: (matrix.wasm || !matrix.cross) && needs.configure.outputs.check-level >= 1
+      - name: Test Summary
+        uses: test-summary/action@v2
+        with:
+          paths: build/stage1/test-results.xml
+        # prefix `if` above with `always` so it's run even if tests failed
+        if: always() && steps.test.conclusion != 'skipped'
+      - name: Check Test Binary
+        run: ${{ matrix.binary-check }} tests/compiler/534.lean.out
+        if: (!matrix.cross) && steps.test.conclusion != 'skipped'
+      - name: Build Stage 2
+        run: |
+          make -C build -j$NPROC stage2
+        if: matrix.test-speedcenter
+      - name: Check Stage 3
+        run: |
+          make -C build -j$NPROC check-stage3
+        if: matrix.test-speedcenter
+      - name: Test Speedcenter Benchmarks
+        run: |
+          # Necessary for some timing metrics but does not work on Namespace runners
+          # and we just want to test that the benchmarks run at all here
+          #echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid
+          export BUILD=$PWD/build PATH=$PWD/build/stage1/bin:$PATH
+          cd tests/bench
+          nix shell .#temci -c temci exec --config speedcenter.yaml --included_blocks fast --runs 1
+        if: matrix.test-speedcenter
+      - name: Check rebootstrap
+        run: |
+          # clean rebuild in case of Makefile changes
+          make -C build update-stage0 && rm -rf build/stage* && make -C build -j$NPROC
+        if: matrix.name == 'Linux' && needs.configure.outputs.check-level >= 1
+      - name: CCache stats
+        run: ccache -s
 
   # This job collects results from all the matrix jobs
-  # This can be made the "required" job, instead of listing each
+  # This can be made the “required” job, instead of listing each
   # matrix job separately
   all-done:
     name: Build matrix complete
@@ -361,6 +481,8 @@ jobs:
 
   # This job creates releases from tags
   # (whether they are "unofficial" releases for experiments, or official releases when the tag is "v" followed by a semver string.)
+  # We do not attempt to automatically construct a changelog here:
+  # unofficial releases don't need them, and official release notes will be written by a human.
   release:
     if: startsWith(github.ref, 'refs/tags/')
     runs-on: ubuntu-latest
@@ -370,7 +492,7 @@ jobs:
         with:
           path: artifacts
       - name: Release
-        uses: softprops/action-gh-release@72f2c25fcb47643c292f7107632f7a47c1df5cd8
+        uses: softprops/action-gh-release@v1
         with:
           files: artifacts/*/*
           fail_on_unmatched_files: true
@@ -414,7 +536,7 @@ jobs:
           echo -e "\n*Full commit log*\n" >> diff.md
           git log --oneline "$last_tag"..HEAD | sed 's/^/* /' >> diff.md
       - name: Release Nightly
-        uses: softprops/action-gh-release@72f2c25fcb47643c292f7107632f7a47c1df5cd8
+        uses: softprops/action-gh-release@v1
         with:
           body_path: diff.md
           prerelease: true
@@ -431,6 +553,6 @@ jobs:
           GITHUB_TOKEN: ${{ secrets.RELEASE_INDEX_TOKEN }}
       - name: Update toolchain on mathlib4's nightly-testing branch
         run: |
-          gh workflow -R leanprover-community/mathlib4-nightly-testing run nightly_bump_toolchain.yml
+          gh workflow -R leanprover-community/mathlib4 run nightly_bump_toolchain.yml
         env:
           GITHUB_TOKEN: ${{ secrets.MATHLIB4_BOT }}

.github/workflows/grove.yml

@@ -1,161 +0,0 @@
-name: Grove
-
-on:
-  workflow_run: # https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_run
-    workflows: [CI]
-    types: [completed]
-
-permissions:
-  pull-requests: write
-
-jobs:
-  grove-build:
-    runs-on: ubuntu-latest
-    if: github.event.workflow_run.conclusion == 'success' && github.repository == 'leanprover/lean4'
-
-    steps:
-      - name: Retrieve information about the original workflow
-        uses: potiuk/get-workflow-origin@v1_1 # https://github.com/marketplace/actions/get-workflow-origin
-        # This action is deprecated and archived, but it seems hard to find a
-        # better solution for getting the PR number
-        # see https://github.com/orgs/community/discussions/25220 for some discussion
-        id: workflow-info
-        with:
-          token: ${{ secrets.GITHUB_TOKEN }}
-          sourceRunId: ${{ github.event.workflow_run.id }}
-
-      - name: Check if should run
-        id: should-run
-        run: |
-          # Check if it's a push to master (no PR number and target branch is master)
-          if [ -z "${{ steps.workflow-info.outputs.pullRequestNumber }}" ]; then
-            if [ "${{ github.event.workflow_run.head_branch }}" = "master" ]; then
-              echo "Push to master detected. Running Grove."
-              echo "should-run=true" >> "$GITHUB_OUTPUT"
-            else
-              echo "Push to non-master branch, skipping"
-              echo "should-run=false" >> "$GITHUB_OUTPUT"
-            fi
-          else
-            # Check if it's a PR with grove label
-            PR_LABELS='${{ steps.workflow-info.outputs.pullRequestLabels }}'
-            if echo "$PR_LABELS" | grep -q '"grove"'; then
-              echo "PR with grove label detected. Running Grove."
-              echo "should-run=true" >> "$GITHUB_OUTPUT"
-            else
-              echo "PR without grove label, skipping"
-              echo "should-run=false" >> "$GITHUB_OUTPUT"
-            fi
-          fi
-
-      - name: Fetch upstream invalidated facts
-        if: ${{ steps.should-run.outputs.should-run == 'true' && steps.workflow-info.outputs.pullRequestNumber != '' }}
-        id: fetch-upstream
-        uses: TwoFx/grove-action/fetch-upstream@v0.4
-        with:
-          artifact-name: grove-invalidated-facts
-          base-ref: master
-
-      - name: Download toolchain for this commit
-        if: ${{ steps.should-run.outputs.should-run == 'true' }}
-        id: download-toolchain
-        uses: dawidd6/action-download-artifact@v11
-        with:
-          commit: ${{ steps.workflow-info.outputs.sourceHeadSha }}
-          workflow: ci.yml
-          path: artifacts
-          name: "build-Linux release"
-          name_is_regexp: true
-
-      - name: Unpack toolchain
-        if: ${{ steps.should-run.outputs.should-run == 'true' }}
-        id: unpack-toolchain
-        run: |
-          cd artifacts
-          # Find the tar.zst file
-          TAR_FILE=$(find . -name "lean-*.tar.zst" -type f | head -1)
-          if [ -z "$TAR_FILE" ]; then
-            echo "Error: No lean-*.tar.zst file found"
-            exit 1
-          fi
-          echo "Found archive: $TAR_FILE"
-
-          # Extract the archive
-          tar --zstd -xf "$TAR_FILE"
-
-          # Find the extracted directory name
-          LEAN_DIR=$(find . -maxdepth 1 -name "lean-*" -type d | head -1)
-          if [ -z "$LEAN_DIR" ]; then
-            echo "Error: No lean-* directory found after extraction"
-            exit 1
-          fi
-          echo "Extracted directory: $LEAN_DIR"
-          echo "lean-dir=$LEAN_DIR" >> "$GITHUB_OUTPUT"
-
-      - name: Build
-        if: ${{ steps.should-run.outputs.should-run == 'true' }}
-        id: build
-        uses: TwoFx/grove-action/build@v0.4
-        with:
-          project-path: doc/std/grove
-          script-name: grove-stdlib
-          invalidated-facts-artifact-name: grove-invalidated-facts
-          comment-artifact-name: grove-comment
-          toolchain-id: lean4
-          toolchain-path: artifacts/${{ steps.unpack-toolchain.outputs.lean-dir }}
-          project-ref: ${{ steps.workflow-info.outputs.sourceHeadSha }}
-
-      # deploy-alias computes a URL component for the PR preview. This
-      # is so we can have a stable name to use for feedback on draft
-      # material.
-      - id: deploy-alias
-        if: ${{ steps.should-run.outputs.should-run == 'true' }}
-        uses: actions/github-script@v7
-        name: Compute Alias
-        with:
-          result-encoding: string
-          script: |
-            if (process.env.PR) {
-                return `pr-${process.env.PR}`
-            } else {
-                return 'deploy-preview-main';
-            }
-        env:
-          PR: ${{ steps.workflow-info.outputs.pullRequestNumber }}
-
-      - name: Deploy to Netlify
-        if: ${{ steps.should-run.outputs.should-run == 'true' }}
-        id: deploy-draft
-        uses: nwtgck/actions-netlify@v3.0
-        with:
-          publish-dir: ${{ steps.build.outputs.out-path }}
-          production-deploy: false
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          alias: ${{ steps.deploy-alias.outputs.result }}
-          enable-commit-comment: false
-          enable-pull-request-comment: false
-          fails-without-credentials: true
-          enable-github-deployment: false
-          enable-commit-status: false
-        env:
-          NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
-          NETLIFY_SITE_ID: "1cacfa39-a11c-467c-99e7-2e01d7b4089e"
-
-      # actions-netlify cannot add deploy links to a PR because it assumes a
-      # pull_request context, not a workflow_run context, see
-      # https://github.com/nwtgck/actions-netlify/issues/545
-      # We work around by using a comment to post the latest link
-      - name: "Comment on PR with preview links"
-        uses: marocchino/sticky-pull-request-comment@v2
-        if: ${{ steps.should-run.outputs.should-run == 'true' && steps.workflow-info.outputs.pullRequestNumber != '' }}
-        with:
-          number: ${{ env.PR_NUMBER }}
-          header: preview-comment
-          recreate: true
-          message: |
-            [Grove](${{ steps.deploy-draft.outputs.deploy-url }}) for revision ${{ steps.workflow-info.outputs.sourceHeadSha }}.
-
-            ${{ steps.build.outputs.comment-text }}
-        env:
-          PR_NUMBER: ${{ steps.workflow-info.outputs.pullRequestNumber }}
-          PR_HEADSHA: ${{ steps.workflow-info.outputs.sourceHeadSha }}

.github/workflows/labels-from-comments.yml

@@ -1,8 +1,7 @@
 # This workflow allows any user to add one of the `awaiting-review`, `awaiting-author`, `WIP`,
-# `release-ci`, or a `changelog-XXX` label by commenting on the PR or issue.
+# or `release-ci` labels by commenting on the PR or issue.
 # If any labels from the set {`awaiting-review`, `awaiting-author`, `WIP`} are added, other labels
 # from that set are removed automatically at the same time.
-# Similarly, if any `changelog-XXX` label is added, other `changelog-YYY` labels are removed.
 
 name: Label PR based on Comment
 
@@ -12,7 +11,7 @@ on:
 
 jobs:
   update-label:
-    if: github.event.issue.pull_request != null && (contains(github.event.comment.body, 'awaiting-review') || contains(github.event.comment.body, 'awaiting-author') || contains(github.event.comment.body, 'WIP') || contains(github.event.comment.body, 'release-ci') || contains(github.event.comment.body, 'changelog-'))
+    if: github.event.issue.pull_request != null && (contains(github.event.comment.body, 'awaiting-review') || contains(github.event.comment.body, 'awaiting-author') || contains(github.event.comment.body, 'WIP') || contains(github.event.comment.body, 'release-ci'))
     runs-on: ubuntu-latest
 
     steps:
@@ -21,14 +20,13 @@ jobs:
       with:
         github-token: ${{ secrets.GITHUB_TOKEN }}
         script: |
-          const { owner, repo, number: issue_number } = context.issue;
+          const { owner, repo, number: issue_number  } = context.issue;
           const commentLines = context.payload.comment.body.split('\r\n');
 
           const awaitingReview = commentLines.includes('awaiting-review');
           const awaitingAuthor = commentLines.includes('awaiting-author');
           const wip = commentLines.includes('WIP');
           const releaseCI = commentLines.includes('release-ci');
-          const changelogMatch = commentLines.find(line => line.startsWith('changelog-'));
 
           if (awaitingReview || awaitingAuthor || wip) {
             await github.rest.issues.removeLabel({ owner, repo, issue_number, name: 'awaiting-review' }).catch(() => {});
@@ -49,19 +47,3 @@ jobs:
           if (releaseCI) {
             await github.rest.issues.addLabels({ owner, repo, issue_number, labels: ['release-ci'] });
           }
-
-          if (changelogMatch) {
-            const changelogLabel = changelogMatch.trim();
-            const { data: existingLabels } = await github.rest.issues.listLabelsOnIssue({ owner, repo, issue_number });
-            const changelogLabels = existingLabels.filter(label => label.name.startsWith('changelog-'));
-
-            // Remove all other changelog labels
-            for (const label of changelogLabels) {
-              if (label.name !== changelogLabel) {
-                await github.rest.issues.removeLabel({ owner, repo, issue_number, name: label.name }).catch(() => {});
-              }
-            }
-
-            // Add the new changelog label
-            await github.rest.issues.addLabels({ owner, repo, issue_number, labels: [changelogLabel] });
-          }

.github/workflows/nix-ci.yml

@@ -0,0 +1,150 @@
+name: Nix CI
+on:
+  push:
+    branches:
+      - master
+    tags:
+      - '*'
+  pull_request:
+  merge_group:
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  # see ci.yml
+  configure:
+    runs-on: ubuntu-latest
+    outputs:
+      matrix: ${{ steps.set-matrix.outputs.result }}
+    steps:
+      - name: Configure build matrix
+        id: set-matrix
+        uses: actions/github-script@v7
+        with:
+          script: |
+            let large = ${{ github.repository == 'leanprover/lean4' }};
+            let matrix = [
+              {
+                "name": "Nix Linux",
+                "os": large ? "nscloud-ubuntu-22.04-amd64-8x8" : "ubuntu-latest",
+              }
+            ];
+            console.log(`matrix:\n${JSON.stringify(matrix, null, 2)}`);
+            return matrix;
+
+  Build:
+    needs: [configure]
+    runs-on: ${{ matrix.os }}
+    defaults:
+      run:
+        shell: nix run .#ciShell -- bash -euxo pipefail {0}
+    strategy:
+      matrix:
+        include: ${{fromJson(needs.configure.outputs.matrix)}}
+      # complete all jobs
+      fail-fast: false
+    name: ${{ matrix.name }}
+    env:
+      NIX_BUILD_ARGS: --print-build-logs --fallback
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          # the default is to use a virtual merge commit between the PR and master: just use the PR
+          ref: ${{ github.event.pull_request.head.sha }}
+      - name: Set Up Nix Cache
+        uses: actions/cache@v4
+        with:
+          path: nix-store-cache
+          key: ${{ matrix.name }}-nix-store-cache-${{ github.sha }}
+          # fall back to (latest) previous cache
+          restore-keys: |
+            ${{ matrix.name }}-nix-store-cache
+          save-always: true
+      - name: Further Set Up Nix Cache
+        shell: bash -euxo pipefail {0}
+        run: |
+          # Nix seems to mutate the cache, so make a copy
+          cp -r nix-store-cache nix-store-cache-copy || true
+      - name: Install Nix
+        uses: DeterminateSystems/nix-installer-action@main
+        with:
+          extra-conf: |
+            extra-sandbox-paths = /nix/var/cache/ccache?
+            substituters = file://${{ github.workspace }}/nix-store-cache-copy?priority=10&trusted=true https://cache.nixos.org  
+      - name: Prepare CCache Cache
+        run: |
+          sudo mkdir -m0770 -p /nix/var/cache/ccache
+          sudo chown -R $USER /nix/var/cache/ccache
+      - name: Setup CCache Cache
+        uses: actions/cache@v4
+        with:
+          path: /nix/var/cache/ccache
+          key: ${{ matrix.name }}-nix-ccache-${{ github.sha }}
+          # fall back to (latest) previous cache
+          restore-keys: |
+            ${{ matrix.name }}-nix-ccache
+          save-always: true
+      - name: Further Set Up CCache Cache
+        run: |
+          sudo chown -R root:nixbld /nix/var/cache
+          sudo chmod -R 770 /nix/var/cache
+      - name: Build
+        run: |
+          nix build $NIX_BUILD_ARGS .#cacheRoots -o push-build
+      - name: Test
+        run: |
+          nix build --keep-failed $NIX_BUILD_ARGS .#test -o push-test || (ln -s /tmp/nix-build-*/source/src/build/ ./push-test; false)
+      - name: Test Summary
+        uses: test-summary/action@v2
+        with:
+          paths: push-test/test-results.xml
+        if: always()
+        continue-on-error: true
+      - name: Build manual
+        run: |
+          nix build $NIX_BUILD_ARGS --update-input lean --no-write-lock-file ./doc#{lean-mdbook,leanInk,alectryon,inked} -o push-doc
+          nix build $NIX_BUILD_ARGS --update-input lean --no-write-lock-file ./doc
+          # https://github.com/netlify/cli/issues/1809
+          cp -r --dereference ./result ./dist
+        if: matrix.name == 'Nix Linux'
+      - name: Check manual for broken links
+        id: lychee
+        uses: lycheeverse/lychee-action@v1.9.0
+        with:
+          fail: false  # report errors but do not block CI on temporary failures
+          # gmplib.org consistently times out from GH actions
+          # the GitHub token is to avoid rate limiting
+          args: --base './dist' --no-progress --github-token ${{ secrets.GITHUB_TOKEN }} --exclude 'gmplib.org' './dist/**/*.html'
+      - name: Rebuild Nix Store Cache
+        run: |
+          rm -rf nix-store-cache || true
+          nix copy ./push-* --to file://$PWD/nix-store-cache?compression=none
+      - id: deploy-info
+        name: Compute Deployment Metadata
+        run: |
+          set -e
+          python3 -c 'import base64; print("alias="+base64.urlsafe_b64encode(bytes.fromhex("${{github.sha}}")).decode("utf-8").rstrip("="))' >> "$GITHUB_OUTPUT"
+          echo "message=`git log -1 --pretty=format:"%s"`" >> "$GITHUB_OUTPUT"
+      - name: Publish manual to Netlify
+        uses: nwtgck/actions-netlify@v2.0
+        id: publish-manual
+        with:
+          publish-dir: ./dist
+          production-branch: master
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          deploy-message: |
+            ${{ github.event_name == 'pull_request' && format('pr#{0}: {1}', github.event.number, github.event.pull_request.title) || format('ref/{0}: {1}', github.ref_name, steps.deploy-info.outputs.message) }}
+          alias: ${{ steps.deploy-info.outputs.alias }}
+          enable-commit-comment: false
+          enable-pull-request-comment: false
+          github-deployment-environment: "lean-lang.org/lean4/doc"
+          fails-without-credentials: false
+        env:
+          NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
+          NETLIFY_SITE_ID: "b8e805d2-7e9b-4f80-91fb-a84d72fc4a68"
+      - name: Fixup CCache Cache
+        run: |
+          sudo chown -R $USER /nix/var/cache

.github/workflows/pr-body.yml

@@ -1,25 +0,0 @@
-name: Check PR body for changelog convention
-
-on:
-  merge_group:
-  pull_request:
-    types: [opened, synchronize, reopened, edited, labeled, converted_to_draft, ready_for_review]
-
-jobs:
-  check-pr-body:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Check PR body
-        if: github.event_name == 'pull_request'
-        uses: actions/github-script@v7
-        with:
-          script: |
-            const { title, body, labels, draft } = context.payload.pull_request;
-            if (!draft && /^(feat|fix):/.test(title) && !labels.some(label => label.name == "changelog-no")) {
-              if (!labels.some(label => label.name.startsWith("changelog-"))) {
-                core.setFailed('feat/fix PR must have a `changelog-*` label');
-              }
-              if (!/^This PR [^<]/.test(body)) {
-                core.setFailed('feat/fix PR must have changelog summary starting with "This PR ..." as first line.');
-              }
-            }

.github/workflows/pr-release.yml

@@ -34,7 +34,7 @@ jobs:
       - name: Download artifact from the previous workflow.
         if: ${{ steps.workflow-info.outputs.pullRequestNumber != '' }}
         id: download-artifact
-        uses: dawidd6/action-download-artifact@v11 # https://github.com/marketplace/actions/download-workflow-artifact
+        uses: dawidd6/action-download-artifact@v2 # https://github.com/marketplace/actions/download-workflow-artifact
         with:
           run_id: ${{ github.event.workflow_run.id }}
           path: artifacts
@@ -48,30 +48,19 @@ jobs:
           git -C lean4.git remote add origin https://github.com/${{ github.repository_owner }}/lean4.git
           git -C lean4.git fetch -n origin master
           git -C lean4.git fetch -n origin "${{ steps.workflow-info.outputs.sourceHeadSha }}"
-          
-          # Create both the original tag and the SHA-suffixed tag
-          SHORT_SHA="${{ steps.workflow-info.outputs.sourceHeadSha }}"
-          SHORT_SHA="${SHORT_SHA:0:7}"
-          
-          # Export the short SHA for use in subsequent steps
-          echo "SHORT_SHA=${SHORT_SHA}" >> "$GITHUB_ENV"
-
           git -C lean4.git tag -f pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }} "${{ steps.workflow-info.outputs.sourceHeadSha }}"
-          git -C lean4.git tag -f pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}-"${SHORT_SHA}" "${{ steps.workflow-info.outputs.sourceHeadSha }}"
-          
           git -C lean4.git remote add pr-releases https://foo:'${{ secrets.PR_RELEASES_TOKEN }}'@github.com/${{ github.repository_owner }}/lean4-pr-releases.git
           git -C lean4.git push -f pr-releases pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}
-          git -C lean4.git push -f pr-releases pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}-"${SHORT_SHA}"
       - name: Delete existing release if present
         if: ${{ steps.workflow-info.outputs.pullRequestNumber != '' }}
         run: |
-          # Try to delete any existing release for the current PR (just the version without the SHA suffix).
+          # Try to delete any existing release for the current PR.
           gh release delete --repo ${{ github.repository_owner }}/lean4-pr-releases pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }} -y || true
         env:
           GH_TOKEN: ${{ secrets.PR_RELEASES_TOKEN }}
-      - name: Release (short format)
+      - name: Release
         if: ${{ steps.workflow-info.outputs.pullRequestNumber != '' }}
-        uses: softprops/action-gh-release@72f2c25fcb47643c292f7107632f7a47c1df5cd8
+        uses: softprops/action-gh-release@v1
         with:
           name: Release for PR ${{ steps.workflow-info.outputs.pullRequestNumber }}
           # There are coredumps files here as well, but all in deeper subdirectories.
@@ -84,24 +73,9 @@ jobs:
           # The token used here must have `workflow` privileges.
           GITHUB_TOKEN: ${{ secrets.PR_RELEASES_TOKEN }}
 
-      - name: Release (SHA-suffixed format)
+      - name: Report release status
         if: ${{ steps.workflow-info.outputs.pullRequestNumber != '' }}
-        uses: softprops/action-gh-release@72f2c25fcb47643c292f7107632f7a47c1df5cd8
-        with:
-          name: Release for PR ${{ steps.workflow-info.outputs.pullRequestNumber }} (${{ steps.workflow-info.outputs.sourceHeadSha }})
-          # There are coredumps files here as well, but all in deeper subdirectories.
-          files: artifacts/*/*
-          fail_on_unmatched_files: true
-          draft: false
-          tag_name: pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}-${{ env.SHORT_SHA }}
-          repository: ${{ github.repository_owner }}/lean4-pr-releases
-        env:
-          # The token used here must have `workflow` privileges.
-          GITHUB_TOKEN: ${{ secrets.PR_RELEASES_TOKEN }}
-
-      - name: Report release status (short format)
-        if: ${{ steps.workflow-info.outputs.pullRequestNumber != '' }}
-        uses: actions/github-script@v7
+        uses: actions/github-script@v6
         with:
           script: |
             await github.rest.repos.createCommitStatus({
@@ -113,20 +87,6 @@ jobs:
               description: "${{ github.repository_owner }}/lean4-pr-releases:pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}",
             });
 
-      - name: Report release status (SHA-suffixed format)
-        if: ${{ steps.workflow-info.outputs.pullRequestNumber != '' }}
-        uses: actions/github-script@v7
-        with:
-          script: |
-            await github.rest.repos.createCommitStatus({
-              owner: context.repo.owner,
-              repo: context.repo.repo,
-              sha: "${{ steps.workflow-info.outputs.sourceHeadSha }}",
-              state: "success",
-              context: "PR toolchain (SHA-suffixed)",
-              description: "${{ github.repository_owner }}/lean4-pr-releases:pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}-${{ env.SHORT_SHA }}",
-            });
-
       - name: Add label
         if: ${{ steps.workflow-info.outputs.pullRequestNumber != '' }}
         uses: actions/github-script@v7
@@ -151,10 +111,10 @@ jobs:
 
       - name: 'Setup jq'
         if: ${{ steps.workflow-info.outputs.pullRequestNumber != '' }}
-        uses: dcarbone/install-jq-action@v3.2.0
+        uses: dcarbone/install-jq-action@v1.0.1
 
       # Check that the most recently nightly coincides with 'git merge-base HEAD master'
-      - name: Check merge-base and nightly-testing-YYYY-MM-DD for Mathlib/Batteries
+      - name: Check merge-base and nightly-testing-YYYY-MM-DD
         if: ${{ steps.workflow-info.outputs.pullRequestNumber != '' }}
         id: ready
         run: |
@@ -167,14 +127,14 @@ jobs:
             echo "The merge base of this PR coincides with the nightly release"
 
             BATTERIES_REMOTE_TAGS="$(git ls-remote https://github.com/leanprover-community/batteries.git nightly-testing-"$MOST_RECENT_NIGHTLY")"
-            MATHLIB_REMOTE_TAGS="$(git ls-remote https://github.com/leanprover-community/mathlib4-nightly-testing.git nightly-testing-"$MOST_RECENT_NIGHTLY")"
+            MATHLIB_REMOTE_TAGS="$(git ls-remote https://github.com/leanprover-community/mathlib4.git nightly-testing-"$MOST_RECENT_NIGHTLY")"
 
             if [[ -n "$BATTERIES_REMOTE_TAGS" ]]; then
               echo "... and Batteries has a 'nightly-testing-$MOST_RECENT_NIGHTLY' tag."
               MESSAGE=""
 
               if [[ -n "$MATHLIB_REMOTE_TAGS" ]]; then
-                echo "... and Mathlib has a 'nightly-testing-$MOST_RECENT_NIGHTLY' tag."
+                echo "... and Mathlib has a 'nightly-testing-$MOST_RECENT_NIGHTLY' tag."    
               else
                 echo "... but Mathlib does not yet have a 'nightly-testing-$MOST_RECENT_NIGHTLY' tag."
                 MESSAGE="- ❗ Mathlib CI can not be attempted yet, as the \`nightly-testing-$MOST_RECENT_NIGHTLY\` tag does not exist there yet. We will retry when you push more commits. If you rebase your branch onto \`nightly-with-mathlib\`, Mathlib CI should run now."
@@ -183,31 +143,18 @@ jobs:
               echo "... but Batteries does not yet have a 'nightly-testing-$MOST_RECENT_NIGHTLY' tag."
               MESSAGE="- ❗ Batteries CI can not be attempted yet, as the \`nightly-testing-$MOST_RECENT_NIGHTLY\` tag does not exist there yet. We will retry when you push more commits. If you rebase your branch onto \`nightly-with-mathlib\`, Batteries CI should run now."
             fi
+
           else
             echo "The most recently nightly tag on this branch has SHA: $NIGHTLY_SHA"
             echo "but 'git merge-base origin/master HEAD' reported: $MERGE_BASE_SHA"
             git -C lean4.git log -10 origin/master
 
-            git -C lean4.git fetch origin nightly-with-mathlib
+            git -C lean4.git fetch origin nightly-with-mathlib 
             NIGHTLY_WITH_MATHLIB_SHA="$(git -C lean4.git rev-parse "origin/nightly-with-mathlib")"
             MESSAGE="- ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the \`nightly-with-mathlib\` branch. Try \`git rebase $MERGE_BASE_SHA --onto $NIGHTLY_WITH_MATHLIB_SHA\`."
           fi
 
           if [[ -n "$MESSAGE" ]]; then
-            # Check if force-mathlib-ci label is present
-            LABELS="$(curl --retry 3 --location --silent \
-                          -H "Authorization: token ${{ secrets.MATHLIB4_COMMENT_BOT }}" \
-                          -H "Accept: application/vnd.github.v3+json" \
-                          "https://api.github.com/repos/leanprover/lean4/issues/${{ steps.workflow-info.outputs.pullRequestNumber }}/labels" \
-                          | jq -r '.[].name')"
-            
-            if echo "$LABELS" | grep -q "^force-mathlib-ci$"; then
-              echo "force-mathlib-ci label detected, forcing CI despite issues"
-              MESSAGE="Forcing Mathlib CI because the \`force-mathlib-ci\` label is present, despite problem: $MESSAGE"
-              FORCE_CI=true
-            else
-              MESSAGE="$MESSAGE You can force Mathlib CI using the \`force-mathlib-ci\` label."
-            fi
 
             echo "Checking existing messages"
 
@@ -217,10 +164,10 @@ jobs:
 
             # Use GitHub API to check if a comment already exists
             existing_comment="$(curl --retry 3 --location --silent \
-                                    -H "Authorization: token ${{ secrets.MATHLIB4_COMMENT_BOT }}" \
+                                    -H "Authorization: token ${{ secrets.MATHLIB4_BOT }}" \
                                     -H "Accept: application/vnd.github.v3+json" \
                                     "https://api.github.com/repos/leanprover/lean4/issues/${{ steps.workflow-info.outputs.pullRequestNumber }}/comments" \
-                                    | jq 'first(.[] | select(.body | test("^- . Mathlib") or startswith("Mathlib CI status")) | select(.user.login == "leanprover-community-bot"))')"
+                                    | jq 'first(.[] | select(.body | test("^- . Mathlib") or startswith("Mathlib CI status")) | select(.user.login == "leanprover-community-mathlib4-bot"))')"
             existing_comment_id="$(echo "$existing_comment" | jq -r .id)"
             existing_comment_body="$(echo "$existing_comment" | jq -r .body)"
 
@@ -230,14 +177,14 @@ jobs:
               echo "Posting message to the comments: $MESSAGE"
 
               # Append new result to the existing comment or post a new comment
-              # It's essential we use the MATHLIB4_COMMENT_BOT token here, so that Mathlib CI can subsequently edit the comment.
+              # It's essential we use the MATHLIB4_BOT token here, so that Mathlib CI can subsequently edit the comment.
               if [ -z "$existing_comment_id" ]; then
                 INTRO="Mathlib CI status ([docs](https://leanprover-community.github.io/contribute/tags_and_branches.html)):"
                 # Post new comment with a bullet point
                 echo "Posting as new comment at leanprover/lean4/issues/${{ steps.workflow-info.outputs.pullRequestNumber }}/comments"
                 curl -L -s \
                   -X POST \
-                  -H "Authorization: token ${{ secrets.MATHLIB4_COMMENT_BOT }}" \
+                  -H "Authorization: token ${{ secrets.MATHLIB4_BOT }}" \
                   -H "Accept: application/vnd.github.v3+json" \
                   -d "$(jq --null-input --arg intro "$INTRO" --arg val "$MESSAGE" '{"body":($intro + "\n" + $val)}')" \
                   "https://api.github.com/repos/leanprover/lean4/issues/${{ steps.workflow-info.outputs.pullRequestNumber }}/comments"
@@ -246,7 +193,7 @@ jobs:
                 echo "Appending to existing comment at leanprover/lean4/issues/${{ steps.workflow-info.outputs.pullRequestNumber }}/comments"
                 curl -L -s \
                   -X PATCH \
-                  -H "Authorization: token ${{ secrets.MATHLIB4_COMMENT_BOT }}" \
+                  -H "Authorization: token ${{ secrets.MATHLIB4_BOT }}" \
                   -H "Accept: application/vnd.github.v3+json" \
                   -d "$(jq --null-input --arg existing "$existing_comment_body" --arg message "$MESSAGE" '{"body":($existing + "\n" + $message)}')" \
                   "https://api.github.com/repos/leanprover/lean4/issues/comments/$existing_comment_id"
@@ -254,121 +201,14 @@ jobs:
             else
               echo "The message already exists in the comment body."
             fi
-
-            if [[ "$FORCE_CI" == "true" ]]; then
-              echo "mathlib_ready=true" >> "$GITHUB_OUTPUT"
-            else
-              echo "mathlib_ready=false" >> "$GITHUB_OUTPUT"
-            fi
+            echo "mathlib_ready=false" >> "$GITHUB_OUTPUT"
           else
             echo "mathlib_ready=true" >> "$GITHUB_OUTPUT"
           fi
 
-      - name: Check merge-base and nightly-testing-YYYY-MM-DD for reference manual
-        if: ${{ steps.workflow-info.outputs.pullRequestNumber != '' }}
-        id: reference-manual-ready
-        run: |
-          echo "Most recent nightly release in your branch: $MOST_RECENT_NIGHTLY"
-          NIGHTLY_SHA=$(git -C lean4.git rev-parse "nightly-$MOST_RECENT_NIGHTLY^{commit}")
-          echo "SHA of most recent nightly release: $NIGHTLY_SHA"
-          MERGE_BASE_SHA=$(git -C lean4.git merge-base origin/master "${{ steps.workflow-info.outputs.sourceHeadSha }}")
-          echo "SHA of merge-base: $MERGE_BASE_SHA"
-          if [ "$NIGHTLY_SHA" = "$MERGE_BASE_SHA" ]; then
-            echo "The merge base of this PR coincides with the nightly release"
-
-            MANUAL_REMOTE_TAGS="$(git ls-remote https://github.com/leanprover/reference-manual.git nightly-testing-"$MOST_RECENT_NIGHTLY")"
-            if [[ -n "$MANUAL_REMOTE_TAGS" ]]; then
-              echo "... and the reference manual has a 'nightly-testing-$MOST_RECENT_NIGHTLY' tag."
-              MESSAGE=""
-            else
-              echo "... but the reference manual does not yet have a 'nightly-testing-$MOST_RECENT_NIGHTLY' tag."
-              MESSAGE="- ❗ Reference manual CI can not be attempted yet, as the \`nightly-testing-$MOST_RECENT_NIGHTLY\` tag does not exist there yet. We will retry when you push more commits. If you rebase your branch onto \`nightly-with-manual\`, reference manual CI should run now."
-            fi
-          else
-            echo "The most recently nightly tag on this branch has SHA: $NIGHTLY_SHA"
-            echo "but 'git merge-base origin/master HEAD' reported: $MERGE_BASE_SHA"
-            git -C lean4.git log -10 origin/master
-
-            git -C lean4.git fetch origin nightly-with-manual
-            NIGHTLY_WITH_MANUAL_SHA="$(git -C lean4.git rev-parse "origin/nightly-with-manual")"
-            MESSAGE="- ❗ Reference manual CI will not be attempted unless your PR branches off the \`nightly-with-manual\` branch. Try \`git rebase $MERGE_BASE_SHA --onto $NIGHTLY_WITH_MANUAL_SHA\`."
-          fi
-
-          if [[ -n "$MESSAGE" ]]; then
-            # Check if force-manual-ci label is present
-            LABELS="$(curl --retry 3 --location --silent \
-                          -H "Authorization: token ${{ secrets.MANUAL_COMMENT_BOT }}" \
-                          -H "Accept: application/vnd.github.v3+json" \
-                          "https://api.github.com/repos/leanprover/lean4/issues/${{ steps.workflow-info.outputs.pullRequestNumber }}/labels" \
-                          | jq -r '.[].name')"
-            
-            if echo "$LABELS" | grep -q "^force-manual-ci$"; then
-              echo "force-manual-ci label detected, forcing CI despite issues"
-              MESSAGE="Forcing reference manual CI because the \`force-manual-ci\` label is present, despite problem: $MESSAGE"
-              FORCE_CI=true
-            else
-              MESSAGE="$MESSAGE You can force reference manual CI using the \`force-manual-ci\` label."
-            fi
-
-            echo "Checking existing messages"
-
-            # The code for updating comments is duplicated in the reference manual's
-            # scripts/lean-pr-testing-comments.sh
-            # so keep in sync
-
-            # Use GitHub API to check if a comment already exists
-            existing_comment="$(curl --retry 3 --location --silent \
-                                    -H "Authorization: token ${{ secrets.MANUAL_COMMENT_BOT }}" \
-                                    -H "Accept: application/vnd.github.v3+json" \
-                                    "https://api.github.com/repos/leanprover/lean4/issues/${{ steps.workflow-info.outputs.pullRequestNumber }}/comments" \
-                                    | jq 'first(.[] | select(.body | test("^- . Manual") or startswith("Reference manual CI status")) | select(.user.login == "leanprover-bot"))')"
-            existing_comment_id="$(echo "$existing_comment" | jq -r .id)"
-            existing_comment_body="$(echo "$existing_comment" | jq -r .body)"
-
-            if [[ "$existing_comment_body" != *"$MESSAGE"* ]]; then
-              MESSAGE="$MESSAGE ($(date "+%Y-%m-%d %H:%M:%S"))"
-
-              echo "Posting message to the comments: $MESSAGE"
-
-              # Append new result to the existing comment or post a new comment
-              # It's essential we use the MANUAL_COMMENT_BOT token here, so that reference manual CI can subsequently edit the comment.
-              if [ -z "$existing_comment_id" ]; then
-                INTRO="Reference manual CI status:"
-                # Post new comment with a bullet point
-                echo "Posting as new comment at leanprover/lean4/issues/${{ steps.workflow-info.outputs.pullRequestNumber }}/comments"
-                curl -L -s \
-                  -X POST \
-                  -H "Authorization: token ${{ secrets.MANUAL_COMMENT_BOT }}" \
-                  -H "Accept: application/vnd.github.v3+json" \
-                  -d "$(jq --null-input --arg intro "$INTRO" --arg val "$MESSAGE" '{"body":($intro + "\n" + $val)}')" \
-                  "https://api.github.com/repos/leanprover/lean4/issues/${{ steps.workflow-info.outputs.pullRequestNumber }}/comments"
-              else
-                # Append new result to the existing comment
-                echo "Appending to existing comment at leanprover/lean4/issues/${{ steps.workflow-info.outputs.pullRequestNumber }}/comments"
-                curl -L -s \
-                  -X PATCH \
-                  -H "Authorization: token ${{ secrets.MANUAL_COMMENT_BOT }}" \
-                  -H "Accept: application/vnd.github.v3+json" \
-                  -d "$(jq --null-input --arg existing "$existing_comment_body" --arg message "$MESSAGE" '{"body":($existing + "\n" + $message)}')" \
-                  "https://api.github.com/repos/leanprover/lean4/issues/comments/$existing_comment_id"
-              fi
-            else
-              echo "The message already exists in the comment body."
-            fi
-
-            if [[ "$FORCE_CI" == "true" ]]; then
-              echo "manual_ready=true" >> "$GITHUB_OUTPUT"
-            else
-              echo "manual_ready=false" >> "$GITHUB_OUTPUT"
-            fi
-          else
-            echo "manual_ready=true" >> "$GITHUB_OUTPUT"
-          fi
-
-
       - name: Report mathlib base
         if: ${{ steps.workflow-info.outputs.pullRequestNumber != '' && steps.ready.outputs.mathlib_ready == 'true' }}
-        uses: actions/github-script@v7
+        uses: actions/github-script@v6
         with:
           script: |
             const description =
@@ -412,7 +252,7 @@ jobs:
           if git ls-remote --heads --tags --exit-code origin "nightly-testing-${MOST_RECENT_NIGHTLY}" >/dev/null; then
             BASE="nightly-testing-${MOST_RECENT_NIGHTLY}"
           else
-            echo "Couldn't find a 'nightly-testing-${MOST_RECENT_NIGHTLY}' tag at Batteries. Falling back to 'nightly-testing'."
+            echo "This shouldn't be possible: couldn't find a 'nightly-testing-${MOST_RECENT_NIGHTLY}' tag at Batteries. Falling back to 'nightly-testing'."
             BASE=nightly-testing
           fi
 
@@ -423,18 +263,16 @@ jobs:
           if [ "$EXISTS" = "0" ]; then
             echo "Branch does not exist, creating it."
             git switch -c lean-pr-testing-${{ steps.workflow-info.outputs.pullRequestNumber }} "$BASE"
-            echo "leanprover/lean4-pr-releases:pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}-${{ env.SHORT_SHA }}" > lean-toolchain
+            echo "leanprover/lean4-pr-releases:pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}" > lean-toolchain
             git add lean-toolchain
             git commit -m "Update lean-toolchain for testing https://github.com/leanprover/lean4/pull/${{ steps.workflow-info.outputs.pullRequestNumber }}"
           else
-            echo "Branch already exists, updating lean-toolchain."
+            echo "Branch already exists, pushing an empty commit."
             git switch lean-pr-testing-${{ steps.workflow-info.outputs.pullRequestNumber }}
             # The Batteries `nightly-testing` or `nightly-testing-YYYY-MM-DD` branch may have moved since this branch was created, so merge their changes.
             # (This should no longer be possible once `nightly-testing-YYYY-MM-DD` is a tag, but it is still safe to merge.)
             git merge "$BASE" --strategy-option ours --no-commit --allow-unrelated-histories
-            echo "leanprover/lean4-pr-releases:pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}-${{ env.SHORT_SHA }}" > lean-toolchain
-            git add lean-toolchain
-            git commit -m "Update lean-toolchain for https://github.com/leanprover/lean4/pull/${{ steps.workflow-info.outputs.pullRequestNumber }}"
+            git commit --allow-empty -m "Trigger CI for https://github.com/leanprover/lean4/pull/${{ steps.workflow-info.outputs.pullRequestNumber }}"
           fi
 
       - name: Push changes
@@ -456,7 +294,7 @@ jobs:
         if: steps.workflow-info.outputs.pullRequestNumber != '' && steps.ready.outputs.mathlib_ready == 'true'
         uses: actions/checkout@v4
         with:
-          repository: leanprover-community/mathlib4-nightly-testing
+          repository: leanprover-community/mathlib4
           token: ${{ secrets.MATHLIB4_BOT }}
           ref: nightly-testing
           fetch-depth: 0 # This ensures we check out all tags and branches.
@@ -478,7 +316,7 @@ jobs:
           if git ls-remote --heads --tags --exit-code origin "nightly-testing-${MOST_RECENT_NIGHTLY}" >/dev/null; then
             BASE="nightly-testing-${MOST_RECENT_NIGHTLY}"
           else
-            echo "Couldn't find a 'nightly-testing-${MOST_RECENT_NIGHTLY}' branch at Mathlib. Falling back to 'nightly-testing'."
+            echo "This shouldn't be possible: couldn't find a 'nightly-testing-${MOST_RECENT_NIGHTLY}' branch at Mathlib. Falling back to 'nightly-testing'."
             BASE=nightly-testing
           fi
 
@@ -489,86 +327,22 @@ jobs:
           if [ "$EXISTS" = "0" ]; then
             echo "Branch does not exist, creating it."
             git switch -c lean-pr-testing-${{ steps.workflow-info.outputs.pullRequestNumber }} "$BASE"
-            echo "leanprover/lean4-pr-releases:pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}-${{ env.SHORT_SHA }}" > lean-toolchain
+            echo "leanprover/lean4-pr-releases:pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}" > lean-toolchain
             git add lean-toolchain
-            sed -i 's,require "leanprover-community" / "batteries" @ git ".\+",require "leanprover-community" / "batteries" @ git "lean-pr-testing-${{ steps.workflow-info.outputs.pullRequestNumber }}",' lakefile.lean
+            sed -i 's,require "leanprover-community" / "batteries" @ git ".\+",require "leanprover-community" / "batteries" @ git "nightly-testing-'"${MOST_RECENT_NIGHTLY}"'",' lakefile.lean
             lake update batteries
             git add lakefile.lean lake-manifest.json
             git commit -m "Update lean-toolchain for testing https://github.com/leanprover/lean4/pull/${{ steps.workflow-info.outputs.pullRequestNumber }}"
           else
-            echo "Branch already exists, updating lean-toolchain and bumping Batteries."
+            echo "Branch already exists, pushing an empty commit."
             git switch lean-pr-testing-${{ steps.workflow-info.outputs.pullRequestNumber }}
             # The Mathlib `nightly-testing` branch or `nightly-testing-YYYY-MM-DD` tag may have moved since this branch was created, so merge their changes.
             # (This should no longer be possible once `nightly-testing-YYYY-MM-DD` is a tag, but it is still safe to merge.)
             git merge "$BASE" --strategy-option ours --no-commit --allow-unrelated-histories
-            echo "leanprover/lean4-pr-releases:pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}-${{ env.SHORT_SHA }}" > lean-toolchain
-            git add lean-toolchain
-            lake update batteries
-            git add lake-manifest.json
-            git commit -m "Update lean-toolchain for https://github.com/leanprover/lean4/pull/${{ steps.workflow-info.outputs.pullRequestNumber }}"
+            git commit --allow-empty -m "Trigger CI for https://github.com/leanprover/lean4/pull/${{ steps.workflow-info.outputs.pullRequestNumber }}"
           fi
 
       - name: Push changes
         if: steps.workflow-info.outputs.pullRequestNumber != '' && steps.ready.outputs.mathlib_ready == 'true'
         run: |
           git push origin lean-pr-testing-${{ steps.workflow-info.outputs.pullRequestNumber }}
-
-      # We next automatically create a reference manual branch using this toolchain.
-      # Reference manual CI will be responsible for reporting back success or failure
-      # to the PR comments asynchronously (and thus transitively SubVerso/Verso).
-      - name: Cleanup workspace
-        if: steps.workflow-info.outputs.pullRequestNumber != '' && steps.reference-manual-ready.outputs.manual_ready == 'true'
-        run: |
-          sudo rm -rf ./*
-
-      # Checkout the reference manual repository with all branches
-      - name: Checkout mathlib4 repository
-        if: steps.workflow-info.outputs.pullRequestNumber != '' && steps.reference-manual-ready.outputs.manual_ready == 'true'
-        uses: actions/checkout@v4
-        with:
-          repository: leanprover/reference-manual
-          token: ${{ secrets.MANUAL_PR_BOT }}
-          ref: nightly-testing
-          fetch-depth: 0 # This ensures we check out all tags and branches.
-
-      - name: Check if tag in reference manual exists
-        if: steps.workflow-info.outputs.pullRequestNumber != '' && steps.reference-manual-ready.outputs.manual_ready == 'true'
-        id: check_manual_tag
-        run: |
-          git config user.name "leanprover-bot"
-          git config user.email "leanprover-bot@lean-fro.org"
-
-          if git ls-remote --heads --tags --exit-code origin "nightly-testing-${MOST_RECENT_NIGHTLY}" >/dev/null; then
-            BASE="nightly-testing-${MOST_RECENT_NIGHTLY}"
-          else
-            echo "Couldn't find a 'nightly-testing-${MOST_RECENT_NIGHTLY}' branch in the reference manual. Falling back to 'nightly-testing'."
-            BASE=nightly-testing
-          fi
-
-          echo "Using base tag: $BASE"
-
-          EXISTS="$(git ls-remote --heads origin lean-pr-testing-${{ steps.workflow-info.outputs.pullRequestNumber }} | wc -l)"
-          echo "Branch exists: $EXISTS"
-          if [ "$EXISTS" = "0" ]; then
-            echo "Branch does not exist, creating it."
-            git switch -c lean-pr-testing-${{ steps.workflow-info.outputs.pullRequestNumber }} "$BASE"
-            echo "leanprover/lean4-pr-releases:pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}-${{ env.SHORT_SHA }}" > lean-toolchain
-            git add lean-toolchain
-            git add lakefile.lean lake-manifest.json
-            git commit -m "Update lean-toolchain for testing https://github.com/leanprover/lean4/pull/${{ steps.workflow-info.outputs.pullRequestNumber }}"
-          else
-            echo "Branch already exists, updating lean-toolchain."
-            git switch lean-pr-testing-${{ steps.workflow-info.outputs.pullRequestNumber }}
-            # The reference manual's `nightly-testing` branch or `nightly-testing-YYYY-MM-DD` tag may have moved since this branch was created, so merge their changes.
-            # (This should no longer be possible once `nightly-testing-YYYY-MM-DD` is a tag, but it is still safe to merge.)
-            git merge "$BASE" --strategy-option ours --no-commit --allow-unrelated-histories
-            echo "leanprover/lean4-pr-releases:pr-release-${{ steps.workflow-info.outputs.pullRequestNumber }}-${{ env.SHORT_SHA }}" > lean-toolchain
-            git add lean-toolchain
-            git add lake-manifest.json
-            git commit -m "Update lean-toolchain for https://github.com/leanprover/lean4/pull/${{ steps.workflow-info.outputs.pullRequestNumber }}"
-          fi
-
-      - name: Push changes
-        if: steps.workflow-info.outputs.pullRequestNumber != '' && steps.reference-manual-ready.outputs.manual_ready == 'true'
-        run: |
-          git push origin lean-pr-testing-${{ steps.workflow-info.outputs.pullRequestNumber }}

.github/workflows/stale.yml

@@ -11,7 +11,7 @@ jobs:
   stale:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/stale@v9
+      - uses: actions/stale@v8
         with:
           days-before-stale: -1
           days-before-pr-stale: 30

.github/workflows/update-stage0.yml

@@ -18,7 +18,7 @@ concurrency:
 
 jobs:
   update-stage0:
-    runs-on: nscloud-ubuntu-22.04-amd64-8x16
+    runs-on: ubuntu-latest
     steps:
     # This action should push to an otherwise protected branch, so it
     # uses a deploy key with write permissions, as suggested at
@@ -40,36 +40,34 @@ jobs:
       run: |
           git config --global user.name "Lean stage0 autoupdater"
           git config --global user.email "<>"
+    # Would be nice, but does not work yet:
+    # https://github.com/DeterminateSystems/magic-nix-cache/issues/39
+    # This action does not run that often and building runs in a few minutes, so ok for now
+    #- if: env.should_update_stage0 == 'yes'
+    #  uses: DeterminateSystems/magic-nix-cache-action@v2
+    - if: env.should_update_stage0 == 'yes'
+      name: Restore Build Cache
+      uses: actions/cache/restore@v4
+      with:
+        path: nix-store-cache
+        key: Nix Linux-nix-store-cache-${{ github.sha }}
+        # fall back to (latest) previous cache
+        restore-keys: |
+          Nix Linux-nix-store-cache
+    - if: env.should_update_stage0 == 'yes'
+      name: Further Set Up Nix Cache
+      shell: bash -euxo pipefail {0}
+      run: |
+        # Nix seems to mutate the cache, so make a copy
+        cp -r nix-store-cache nix-store-cache-copy || true
     - if: env.should_update_stage0 == 'yes'
       name: Install Nix
       uses: DeterminateSystems/nix-installer-action@main
-    - name: Open Nix shell once
-      if: env.should_update_stage0 == 'yes'
-      run: true
-      shell: 'nix develop -c bash -euxo pipefail {0}'
-    - name: Set up NPROC
-      if: env.should_update_stage0 == 'yes'
-      run: |
-        echo "NPROC=$(nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 4)" >> $GITHUB_ENV
-      shell: 'nix develop -c bash -euxo pipefail {0}'
-    - name: Restore Cache
-      if: env.should_update_stage0 == 'yes'
-      uses: actions/cache/restore@v4
       with:
-        # NOTE: must be in sync with `restore-cache` in `build-template.yml`
-        # TODO: actually switch to USE_LAKE once it caches more; for now it just caches more often than any other build type so let's use its cache
-        path: |
-          .ccache
-        key: Linux Lake-build-v3-${{ github.sha }}
-        # fall back to (latest) previous cache
-        restore-keys: |
-          Linux Lake-build-v3
+        extra-conf: |
+          substituters = file://${{ github.workspace }}/nix-store-cache-copy?priority=10&trusted=true https://cache.nixos.org  
     - if: env.should_update_stage0 == 'yes'
-      run: cmake --preset release
-      shell: 'nix develop -c bash -euxo pipefail {0}'
-    - if: env.should_update_stage0 == 'yes'
-      run: make -j$NPROC -C build/release  update-stage0-commit
-      shell: 'nix develop -c bash -euxo pipefail {0}'
+      run: nix run .#update-stage0-commit
     - if: env.should_update_stage0 == 'yes'
       run: git show --stat
     - if: env.should_update_stage0 == 'yes' && github.event_name == 'push'

.gitignore

@@ -6,6 +6,7 @@
 lake-manifest.json
 /build
 /src/lakefile.toml
+/tests/lakefile.toml
 /lakefile.toml
 GPATH
 GRTAGS
@@ -21,7 +22,6 @@ settings.json
 .gdb_history
 .vscode/*
 !.vscode/settings.json
-script/__pycache__
 *.produced.out
 CMakeSettings.json
 CppProperties.json
@@ -31,4 +31,3 @@ fwOut.txt
 wdErr.txt
 wdIn.txt
 wdOut.txt
-downstream_releases/

.gitpod.Dockerfile

@@ -1,14 +0,0 @@
-# You can find the new timestamped tags here: https://hub.docker.com/r/gitpod/workspace-full/tags
-FROM gitpod/workspace-full
-
-USER root
-RUN apt-get update && apt-get install git libgmp-dev libuv1-dev cmake ccache clang -y && apt-get clean
-
-USER gitpod
-
-# Install and configure elan
-RUN curl https://raw.githubusercontent.com/leanprover/elan/master/elan-init.sh -sSf | sh -s -- -y --default-toolchain none
-ENV PATH="/home/gitpod/.elan/bin:${PATH}"
-# Create a dummy toolchain so that we can pre-register it with elan
-RUN mkdir -p /workspace/lean4/build/release/stage1/bin && touch /workspace/lean4/build/release/stage1/bin/lean && elan toolchain link lean4 /workspace/lean4/build/release/stage1
-RUN mkdir -p /workspace/lean4/build/release/stage0/bin && touch /workspace/lean4/build/release/stage0/bin/lean && elan toolchain link lean4-stage0 /workspace/lean4/build/release/stage0

.gitpod.yml

@@ -1,11 +0,0 @@
-image:
-    file: .gitpod.Dockerfile
-
-vscode:
-  extensions:
-    - leanprover.lean4
-
-tasks:
-  - name: Release build
-    init: cmake --preset release
-    command: make -C build/release -j$(nproc || sysctl -n hw.logicalcpu)

CMakeLists.txt

@@ -1,30 +1,23 @@
 cmake_minimum_required(VERSION 3.11)
-
-option(USE_MIMALLOC "use mimalloc" ON)
-
 # store all variables passed on the command line into CL_ARGS so we can pass them to the stage builds
 # https://stackoverflow.com/a/48555098/161659
 # MUST be done before call to 'project'
-# Use standard release build (discarding LEAN_EXTRA_CXX_FLAGS etc.) for stage0 by default since it is assumed to be "good", but still pass through CMake platform arguments (compiler, toolchain file, ..).
+# Use standard release build (discarding LEAN_CXX_EXTRA_FLAGS etc.) for stage0 by default since it is assumed to be "good", but still pass through CMake platform arguments (compiler, toolchain file, ..).
 # Use `STAGE0_` prefix to pass variables to stage0 explicitly.
 get_cmake_property(vars CACHE_VARIABLES)
 foreach(var ${vars})
   get_property(currentHelpString CACHE "${var}" PROPERTY HELPSTRING)
   if("${var}" MATCHES "STAGE0_(.*)")
     list(APPEND STAGE0_ARGS "-D${CMAKE_MATCH_1}=${${var}}")
-  elseif("${var}" MATCHES "STAGE1_(.*)")
-    list(APPEND STAGE1_ARGS "-D${CMAKE_MATCH_1}=${${var}}")
   elseif("${currentHelpString}" MATCHES "No help, variable specified on the command line." OR "${currentHelpString}" STREQUAL "")
     list(APPEND CL_ARGS "-D${var}=${${var}}")
-    if("${var}" MATCHES "USE_GMP|CHECK_OLEAN_VERSION|LEAN_VERSION_.*|LEAN_SPECIAL_VERSION_DESC")
+    if("${var}" MATCHES "USE_GMP|CHECK_OLEAN_VERSION")
       # must forward options that generate incompatible .olean format
       list(APPEND STAGE0_ARGS "-D${var}=${${var}}")
-    elseif("${var}" MATCHES "LLVM*|PKG_CONFIG|USE_LAKE|USE_MIMALLOC")
+    endif()
+    if("${var}" MATCHES "LLVM*")
       list(APPEND STAGE0_ARGS "-D${var}=${${var}}")
     endif()
-  elseif("${var}" MATCHES "USE_MIMALLOC")
-    list(APPEND CL_ARGS "-D${var}=${${var}}")
-    list(APPEND STAGE0_ARGS "-D${var}=${${var}}")
   elseif(("${var}" MATCHES "CMAKE_.*") AND NOT ("${var}" MATCHES "CMAKE_BUILD_TYPE") AND NOT ("${var}" MATCHES "CMAKE_HOME_DIRECTORY"))
     list(APPEND PLATFORM_ARGS "-D${var}=${${var}}")
   endif()
@@ -39,14 +32,10 @@ endif()
 
 # Don't do anything with cadical on wasm
 if (NOT ${CMAKE_SYSTEM_NAME} MATCHES "Emscripten")
+  # On CI Linux, we source cadical from Nix instead; see flake.nix
   find_program(CADICAL cadical)
   if(NOT CADICAL)
     set(CADICAL_CXX c++)
-    if (CADICAL_USE_CUSTOM_CXX)
-      set(CADICAL_CXX ${CMAKE_CXX_COMPILER})
-      set(CADICAL_CXXFLAGS "${LEAN_EXTRA_CXX_FLAGS}")
-      set(CADICAL_LDFLAGS "-Wl,-rpath=\\$$ORIGIN/../lib")
-    endif()
     find_program(CCACHE ccache)
     if(CCACHE)
       set(CADICAL_CXX "${CCACHE} ${CADICAL_CXX}")
@@ -55,61 +44,41 @@ if (NOT ${CMAKE_SYSTEM_NAME} MATCHES "Emscripten")
     if(${CMAKE_SYSTEM_NAME} MATCHES "Windows")
       string(APPEND CADICAL_CXXFLAGS " -DNUNLOCKED")
     endif()
-    string(APPEND CADICAL_CXXFLAGS " -DNCLOSEFROM")
     ExternalProject_add(cadical
       PREFIX cadical
       GIT_REPOSITORY https://github.com/arminbiere/cadical
-      GIT_TAG rel-2.1.2
+      GIT_TAG rel-1.9.5
       CONFIGURE_COMMAND ""
-      BUILD_COMMAND $(MAKE) -f ${CMAKE_SOURCE_DIR}/src/cadical.mk
-        CMAKE_EXECUTABLE_SUFFIX=${CMAKE_EXECUTABLE_SUFFIX}
-        CXX=${CADICAL_CXX}
-        CXXFLAGS=${CADICAL_CXXFLAGS}
-        LDFLAGS=${CADICAL_LDFLAGS}
+      # https://github.com/arminbiere/cadical/blob/master/BUILD.md#manual-build
+      BUILD_COMMAND $(MAKE) -f ${CMAKE_SOURCE_DIR}/src/cadical.mk CMAKE_EXECUTABLE_SUFFIX=${CMAKE_EXECUTABLE_SUFFIX} CXX=${CADICAL_CXX} CXXFLAGS=${CADICAL_CXXFLAGS}
       BUILD_IN_SOURCE ON
       INSTALL_COMMAND "")
     set(CADICAL ${CMAKE_BINARY_DIR}/cadical/cadical${CMAKE_EXECUTABLE_SUFFIX} CACHE FILEPATH "path to cadical binary" FORCE)
-    list(APPEND EXTRA_DEPENDS cadical)
+    set(EXTRA_DEPENDS "cadical")
   endif()
   list(APPEND CL_ARGS -DCADICAL=${CADICAL})
 endif()
 
-if (USE_MIMALLOC)
-  ExternalProject_add(mimalloc
-    PREFIX mimalloc
-    GIT_REPOSITORY https://github.com/microsoft/mimalloc
-    GIT_TAG v2.2.3
-    # just download, we compile it as part of each stage as it is small
-    CONFIGURE_COMMAND ""
-    BUILD_COMMAND ""
-    INSTALL_COMMAND "")
-  list(APPEND EXTRA_DEPENDS mimalloc)
-endif()
-
-if (NOT STAGE1_PREV_STAGE)
-  ExternalProject_add(stage0
-    SOURCE_DIR "${LEAN_SOURCE_DIR}/stage0"
-    SOURCE_SUBDIR src
-    BINARY_DIR stage0
-    # do not rebuild stage0 when git hash changes; it's not from this commit anyway
-    # (however, CI will override this as we need to embed the githash into the stage 1 library built
-    # by stage 0)
-    CMAKE_ARGS -DSTAGE=0 -DUSE_GITHASH=OFF ${PLATFORM_ARGS} ${STAGE0_ARGS}
-    BUILD_ALWAYS ON  # cmake doesn't auto-detect changes without a download method
-    INSTALL_COMMAND ""  # skip install
-    DEPENDS ${EXTRA_DEPENDS}
-  )
-  list(APPEND EXTRA_DEPENDS stage0)
-endif()
+ExternalProject_add(stage0
+  SOURCE_DIR "${LEAN_SOURCE_DIR}/stage0"
+  SOURCE_SUBDIR src
+  BINARY_DIR stage0
+  # do not rebuild stage0 when git hash changes; it's not from this commit anyway
+  # (however, `CHECK_OLEAN_VERSION=ON` in CI will override this as we need to
+  # embed the githash into the stage 1 library built by stage 0)
+  CMAKE_ARGS -DSTAGE=0 -DUSE_GITHASH=OFF ${PLATFORM_ARGS} ${STAGE0_ARGS}
+  BUILD_ALWAYS ON  # cmake doesn't auto-detect changes without a download method
+  INSTALL_COMMAND ""  # skip install
+  DEPENDS ${EXTRA_DEPENDS}
+)
 ExternalProject_add(stage1
   SOURCE_DIR "${LEAN_SOURCE_DIR}"
   SOURCE_SUBDIR src
   BINARY_DIR stage1
-  CMAKE_ARGS -DSTAGE=1 -DPREV_STAGE=${CMAKE_BINARY_DIR}/stage0 -DPREV_STAGE_CMAKE_EXECUTABLE_SUFFIX=${STAGE0_CMAKE_EXECUTABLE_SUFFIX} ${CL_ARGS} ${STAGE1_ARGS}
+  CMAKE_ARGS -DSTAGE=1 -DPREV_STAGE=${CMAKE_BINARY_DIR}/stage0 -DPREV_STAGE_CMAKE_EXECUTABLE_SUFFIX=${STAGE0_CMAKE_EXECUTABLE_SUFFIX} ${CL_ARGS}
   BUILD_ALWAYS ON
   INSTALL_COMMAND ""
-  DEPENDS ${EXTRA_DEPENDS}
-  STEP_TARGETS configure
+  DEPENDS stage0
 )
 ExternalProject_add(stage2
   SOURCE_DIR "${LEAN_SOURCE_DIR}"
@@ -120,7 +89,6 @@ ExternalProject_add(stage2
   INSTALL_COMMAND ""
   DEPENDS stage1
   EXCLUDE_FROM_ALL ON
-  STEP_TARGETS configure
 )
 ExternalProject_add(stage3
   SOURCE_DIR "${LEAN_SOURCE_DIR}"

CMakePresets.json

@@ -16,39 +16,26 @@
       "name": "debug",
       "displayName": "Debug build config",
       "cacheVariables": {
-        "LEAN_EXTRA_CXX_FLAGS": "-DLEAN_DEFAULT_THREAD_STACK_SIZE=16*1024*1024",
         "CMAKE_BUILD_TYPE": "Debug"
       },
       "generator": "Unix Makefiles",
       "binaryDir": "${sourceDir}/build/debug"
     },
-    {
-      "name": "reldebug",
-      "displayName": "Release with assertions enabled",
-      "cacheVariables": {
-        "CMAKE_BUILD_TYPE": "RelWithAssert"
-      },
-      "generator": "Unix Makefiles",
-      "binaryDir": "${sourceDir}/build/reldebug"
-    },
     {
       "name": "sanitize",
       "displayName": "Sanitize build config",
       "cacheVariables": {
-        "LEAN_EXTRA_CXX_FLAGS": "-fsanitize=address,undefined -DLEAN_DEFAULT_THREAD_STACK_SIZE=16*1024*1024",
-        "LEANC_EXTRA_CC_FLAGS": "-fsanitize=address,undefined",
-        "LEAN_EXTRA_LINKER_FLAGS": "-fsanitize=address,undefined -fsanitize-link-c++-runtime",
+        "LEAN_EXTRA_CXX_FLAGS": "-fsanitize=address,undefined",
+        "LEANC_EXTRA_FLAGS": "-fsanitize=address,undefined -fsanitize-link-c++-runtime",
         "SMALL_ALLOCATOR": "OFF",
-        "USE_MIMALLOC": "OFF",
-        "BSYMBOLIC": "OFF",
-        "LEAN_TEST_VARS": "MAIN_STACK_SIZE=16000"
+        "BSYMBOLIC": "OFF"
       },
       "generator": "Unix Makefiles",
       "binaryDir": "${sourceDir}/build/sanitize"
     },
     {
       "name": "sandebug",
-      "inherits": ["sanitize", "debug"],
+      "inherits": ["debug", "sanitize"],
       "displayName": "Sanitize+debug build config",
       "binaryDir": "${sourceDir}/build/sandebug"
     }
@@ -62,10 +49,6 @@
       "name": "debug",
       "configurePreset": "debug"
     },
-    {
-      "name": "reldebug",
-      "configurePreset": "reldebug"
-    },
     {
       "name": "sanitize",
       "configurePreset": "sanitize"
@@ -86,11 +69,6 @@
       "configurePreset": "debug",
       "inherits": "release"
     },
-    {
-      "name": "reldebug",
-      "configurePreset": "reldebug",
-      "inherits": "release"
-    },
     {
       "name": "sanitize",
       "configurePreset": "sanitize",

CODEOWNERS

@@ -4,21 +4,22 @@
 # Listed persons will automatically be asked by GitHub to review a PR touching these paths.
 # If multiple names are listed, a review by any of them is considered sufficient by default.
 
-/.github/ @kim-em
-/RELEASES.md @kim-em
+/.github/ @Kha @semorrison
+/RELEASES.md @semorrison
 /src/kernel/ @leodemoura
-/src/library/compiler/ @zwarich
 /src/lake/ @tydeu
-/src/Lean/Compiler/ @leodemoura @zwarich
+/src/Lean/Compiler/ @leodemoura
 /src/Lean/Data/Lsp/ @mhuisi
-/src/Lean/Elab/Deriving/ @kim-em
-/src/Lean/Elab/Tactic/ @kim-em
+/src/Lean/Elab/Deriving/ @semorrison
+/src/Lean/Elab/Tactic/ @semorrison
 /src/Lean/Language/ @Kha
 /src/Lean/Meta/Tactic/ @leodemoura
-/src/Lean/PrettyPrinter/ @kmill
+/src/Lean/Parser/ @Kha
+/src/Lean/PrettyPrinter/ @Kha
+/src/Lean/PrettyPrinter/Delaborator/ @kmill
 /src/Lean/Server/ @mhuisi
 /src/Lean/Widget/ @Vtec234
-/src/Init/Data/ @kim-em
+/src/Init/Data/ @semorrison
 /src/Init/Data/Array/Lemmas.lean @digama0
 /src/Init/Data/List/Lemmas.lean @digama0
 /src/Init/Data/List/BasicAux.lean @digama0
@@ -44,11 +45,3 @@
 /src/Std/ @TwoFX
 /src/Std/Tactic/BVDecide/ @hargoniX
 /src/Lean/Elab/Tactic/BVDecide/ @hargoniX
-/src/Std/Sat/ @hargoniX
-/src/Std/Do @sgraf812
-/src/Std/Tactic/Do @sgraf812
-/src/Lean/Elab/Tactic/Do @sgraf812
-/src/Init/Data/Range/Polymorphic @datokrat
-/src/Init/Data/Slice @datokrat
-/src/Init/Data/Iterators @datokrat
-/src/Std/Data/Iterators @datokrat

README.md

@@ -2,19 +2,19 @@ This is the repository for **Lean 4**.
 
 # About
 
-- [Quickstart](https://lean-lang.org/documentation/setup/)
+- [Quickstart](https://lean-lang.org/lean4/doc/quickstart.html)
 - [Homepage](https://lean-lang.org)
 - [Theorem Proving Tutorial](https://lean-lang.org/theorem_proving_in_lean4/)
 - [Functional Programming in Lean](https://lean-lang.org/functional_programming_in_lean/)
-- [Documentation Overview](https://lean-lang.org/documentation/)
-- [Language Reference](https://lean-lang.org/doc/reference/latest/)
+- [Manual](https://lean-lang.org/lean4/doc/)
 - [Release notes](RELEASES.md) starting at v4.0.0-m3
-- [Examples](https://lean-lang.org/documentation/examples/)
+- [Examples](https://lean-lang.org/lean4/doc/examples.html)
 - [External Contribution Guidelines](CONTRIBUTING.md)
+- [FAQ](https://lean-lang.org/lean4/doc/faq.html)
 
 # Installation
 
-See [Setting Up Lean](https://lean-lang.org/documentation/setup/).
+See [Setting Up Lean](https://lean-lang.org/lean4/doc/setup.html).
 
 # Contributing
 
@@ -22,4 +22,4 @@ Please read our [Contribution Guidelines](CONTRIBUTING.md) first.
 
 # Building from Source
 
-See [Building Lean](doc/make/index.md).
+See [Building Lean](https://lean-lang.org/lean4/doc/make/index.html) (documentation source: [doc/make/index.md](doc/make/index.md)).

debug.log

@@ -0,0 +1 @@
+[0829/202002.254:ERROR:crashpad_client_win.cc(868)] not connected

doc/README.md

@@ -1,10 +0,0 @@
-# Developer Documentation and Examples
-
-This directory contains documentation that describes how to work on
-Lean itself, as well as examples that are included in documentation
-that's hosted on the Lean website.  The `make` directory contains
-information on building Lean, and the `dev` directory describes how to
-work on Lean.
-
-The [documentation section](https://lean-lang.org/documentation) has
-links to documentation that describes how to use Lean itself.

doc/SUMMARY.md

@@ -0,0 +1,94 @@
+# Summary
+
+- [What is Lean](./whatIsLean.md)
+- [Tour of Lean](./tour.md)
+- [Setting Up Lean](./quickstart.md)
+  - [Extended Setup Notes](./setup.md)
+- [Theorem Proving in Lean](./tpil.md)
+- [Functional Programming in Lean](fplean.md)
+- [Examples](./examples.md)
+  - [Palindromes](examples/palindromes.lean.md)
+  - [Binary Search Trees](examples/bintree.lean.md)
+  - [A Certified Type Checker](examples/tc.lean.md)
+  - [The Well-Typed Interpreter](examples/interp.lean.md)
+  - [Dependent de Bruijn Indices](examples/deBruijn.lean.md)
+  - [Parametric Higher-Order Abstract Syntax](examples/phoas.lean.md)
+
+# Language Manual
+<!-- - [Using Lean](./using_lean.md) -->
+<!-- - [Lexical Structure](./lexical_structure.md) -->
+<!-- - [Expressions](./expressions.md) -->
+<!-- - [Declarations](./declarations.md) -->
+- [Organizational features](./organization.md)
+  - [Sections](./sections.md)
+  - [Namespaces](./namespaces.md)
+  - [Implicit Arguments](./implicit.md)
+  - [Auto Bound Implicit Arguments](./autobound.md)
+<!-- - [Dependent Types](./deptypes.md) -->
+<!--   - [Simple Type Theory](./simptypes.md) -->
+<!--   - [Types as objects](./typeobjs.md) -->
+<!--   - [Function Abstraction and Evaluation](./funabst.md) -->
+<!--   - [Introducing Definitions](./introdef.md) -->
+<!--   - [What makes dependent type theory dependent?](./dep.md) -->
+<!-- - [Tactics](./tactics.md) -->
+- [Syntax Extensions](./syntax.md)
+  - [The `do` Notation](./do.md)
+  - [String Interpolation](./stringinterp.md)
+  - [User-Defined Notation](./notation.md)
+  - [Macro Overview](./macro_overview.md)
+  - [Elaborators](./elaborators.md)
+  - [Examples](./syntax_examples.md)
+    - [Balanced Parentheses](./syntax_example.md)
+    - [Arithmetic DSL](./metaprogramming-arith.md)
+- [Declaring New Types](./decltypes.md)
+  - [Enumerated Types](./enum.md)
+  - [Inductive Types](./inductive.md)
+  - [Structures](./struct.md)
+  - [Type classes](./typeclass.md)
+  - [Unification Hints](./unifhint.md)
+- [Builtin Types](./builtintypes.md)
+  - [Natural number](./nat.md)
+  - [Integer](./int.md)
+  - [Fixed precision unsigned integer](./uint.md)
+  - [Float](./float.md)
+  - [Array](./array.md)
+  - [List](./list.md)
+  - [Character](./char.md)
+  - [String](./string.md)
+  - [Option](./option.md)
+  - [Thunk](./thunk.md)
+  - [Task and Thread](./task.md)
+- [Functions](./functions.md)
+- [Monads](./monads/intro.md)
+  - [Functor](./monads/functors.lean.md)
+  - [Applicative](./monads/applicatives.lean.md)
+  - [Monad](./monads/monads.lean.md)
+  - [Reader](./monads/readers.lean.md)
+  - [State](./monads/states.lean.md)
+  - [Except](./monads/except.lean.md)
+  - [Transformers](./monads/transformers.lean.md)
+  - [Laws](./monads/laws.lean.md)
+
+# Other
+
+- [Frequently Asked Questions](./faq.md)
+- [Significant Changes from Lean 3](./lean3changes.md)
+- [Syntax Highlighting Lean in LaTeX](./syntax_highlight_in_latex.md)
+- [User Widgets](examples/widgets.lean.md)
+- [Semantic Highlighting](./semantic_highlighting.md)
+
+# Development
+
+- [Development Guide](./dev/index.md)
+- [Building Lean](./make/index.md)
+  - [Ubuntu Setup](./make/ubuntu.md)
+  - [macOS Setup](./make/osx-10.9.md)
+  - [Windows MSYS2 Setup](./make/msys2.md)
+  - [Windows with WSL](./make/wsl.md)
+- [Bootstrapping](./dev/bootstrap.md)
+- [Testing](./dev/testing.md)
+- [Debugging](./dev/debugging.md)
+- [Commit Convention](./dev/commit_convention.md)
+- [Release checklist](./dev/release_checklist.md)
+- [Building This Manual](./dev/mdbook.md)
+- [Foreign Function Interface](./dev/ffi.md)

doc/alectryon.css

@@ -0,0 +1,786 @@
+@charset "UTF-8";
+/*
+Copyright © 2019 Clément Pit-Claudel
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+*/
+
+/*******************************/
+/* CSS reset for .alectryon-io */
+/*******************************/
+
+.content {
+    /*
+    Use `initial` instead of `contents` to avoid a browser bug which removes
+    the element from the accessibility tree.
+    https://developer.mozilla.org/en-US/docs/Web/CSS/display#display_contents
+     */
+    display: initial;
+}
+
+.alectryon-io blockquote {
+    line-height: inherit;
+}
+
+.alectryon-io blockquote:after {
+    display: none;
+}
+
+.alectryon-io label {
+    display: inline;
+    font-size: inherit;
+    margin: 0;
+}
+
+.alectryon-io a {
+    text-decoration: none !important;
+    font-style: oblique !important;
+    color: unset;
+}
+
+/* Undo <small> and <blockquote>, added to improve RSS rendering. */
+
+.alectryon-io small.alectryon-output,
+.alectryon-io small.alectryon-type-info {
+    font-size: inherit;
+}
+
+.alectryon-io blockquote.alectryon-goal,
+.alectryon-io blockquote.alectryon-message {
+    font-weight: normal;
+    font-size: inherit;
+}
+
+/***************/
+/* Main styles */
+/***************/
+
+.alectryon-coqdoc .doc .code,
+.alectryon-coqdoc .doc .comment,
+.alectryon-coqdoc .doc .inlinecode,
+.alectryon-mref,
+.alectryon-block, .alectryon-io,
+.alectryon-toggle-label, .alectryon-banner {
+    font-family: "Source Code Pro", Consolas, "Ubuntu Mono", Menlo, "DejaVu Sans Mono", monospace, monospace !important;
+    font-size: 0.875em;
+    font-feature-settings: "COQX" 1 /* Coq ligatures */, "XV00" 1 /* Legacy */, "calt" 1 /* Fallback */;
+    line-height: initial;
+}
+
+.alectryon-io, .alectryon-block, .alectryon-toggle-label, .alectryon-banner {
+    overflow: visible;
+    overflow-wrap: break-word;
+    position: relative;
+    white-space: pre-wrap;
+}
+
+/*
+CoqIDE doesn't turn off the unicode bidirectional algorithm (and PG simply
+respects the user's `bidi-display-reordering` setting), so don't turn it off
+here either.  But beware unexpected results like `Definition test_אב := 0.`
+
+.alectryon-io span {
+    direction: ltr;
+    unicode-bidi: bidi-override;
+}
+
+In any case, make an exception for comments:
+
+.highlight .c {
+    direction: embed;
+    unicode-bidi: initial;
+}
+*/
+
+.alectryon-mref,
+.alectryon-mref-marker {
+    align-self: center;
+    box-sizing: border-box;
+    display: inline-block;
+    font-size: 80%;
+    font-weight: bold;
+    line-height: 1;
+    box-shadow: 0 0 0 1pt black;
+    padding: 1pt 0.3em;
+    text-decoration: none;
+}
+
+.alectryon-block .alectryon-mref-marker,
+.alectryon-io .alectryon-mref-marker {
+    user-select: none;
+    margin: -0.25em 0 -0.25em 0.5em;
+}
+
+.alectryon-inline .alectryon-mref-marker {
+    margin: -0.25em 0.15em -0.25em 0.625em; /* 625 = 0.5em / 80% */
+}
+
+.alectryon-mref {
+    color: inherit;
+    margin: -0.5em 0.25em;
+}
+
+.alectryon-goal:target .goal-separator .alectryon-mref-marker,
+:target > .alectryon-mref-marker {
+    animation: blink 0.2s step-start 0s 3 normal none;
+    background-color: #fcaf3e;
+    position: relative;
+}
+
+@keyframes blink {
+    50% {
+        box-shadow: 0 0 0 3pt #fcaf3e, 0 0 0 4pt black;
+        z-index: 10;
+    }
+}
+
+.alectryon-toggle,
+.alectryon-io .alectryon-extra-goal-toggle {
+    display: none;
+}
+
+.alectryon-bubble,
+.alectryon-io label,
+.alectryon-toggle-label {
+    cursor: pointer;
+}
+
+.alectryon-toggle-label {
+    display: block;
+    font-size: 0.8em;
+}
+
+.alectryon-io .alectryon-input {
+    padding: 0.1em 0; /* Enlarge the hitbox slightly to fill interline gaps */
+}
+
+.alectryon-io .alectryon-token {
+    white-space: pre-wrap;
+    display: inline;
+}
+
+.alectryon-io .alectryon-sentence.alectryon-target .alectryon-input {
+    /* FIXME if keywords were ‘bolder’ we wouldn't need !important */
+    font-weight: bold !important; /* Use !important to avoid a * selector */
+}
+
+.alectryon-bubble:before,
+.alectryon-toggle-label:before,
+.alectryon-io label.alectryon-input:after,
+.alectryon-io .alectryon-goal > label:before {
+    border: 1px solid #babdb6;
+    border-radius: 1em;
+    box-sizing: border-box;
+    content: '';
+    display: inline-block;
+    font-weight: bold;
+    height: 0.25em;
+    margin-bottom: 0.15em;
+    vertical-align: middle;
+    width: 0.75em;
+}
+
+.alectryon-toggle-label:before,
+.alectryon-io .alectryon-goal > label:before {
+    margin-right: 0.25em;
+}
+
+.alectryon-io .alectryon-goal > label:before {
+    margin-top: 0.125em;
+}
+
+.alectryon-io label.alectryon-input {
+    padding-right: 1em; /* Prevent line wraps before the checkbox bubble */
+}
+
+.alectryon-io label.alectryon-input:after {
+    margin-left: 0.25em;
+    margin-right: -1em; /* Compensate for the anti-wrapping space */
+}
+
+.alectryon-failed {
+    /* Underlines are broken in Chrome (they reset at each element boundary)… */
+    /* text-decoration: red wavy underline; */
+    /* … but it isn't too noticeable with dots */
+    text-decoration: red dotted underline;
+    text-decoration-skip-ink: none;
+    /* Chrome prints background images in low resolution, yielding a blurry underline */
+    /* background: bottom / 0.3em auto repeat-x url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyLjY0NiAxLjg1MiIgaGVpZ2h0PSI4IiB3aWR0aD0iMTAiPjxwYXRoIGQ9Ik0wIC4yNjVjLjc5NCAwIC41MyAxLjMyMiAxLjMyMyAxLjMyMi43OTQgMCAuNTMtMS4zMjIgMS4zMjMtMS4zMjIiIGZpbGw9Im5vbmUiIHN0cm9rZT0icmVkIiBzdHJva2Utd2lkdGg9Ii41MjkiLz48L3N2Zz4=); */
+}
+
+/* Wrapping :hover rules in a media query ensures that tapping a Coq sentence
+   doesn't trigger its :hover state (otherwise, on mobile, tapping a sentence to
+   hide its output causes it to remain visible (its :hover state gets triggered.
+   We only do it for the default style though, since other styles don't put the
+   output over the main text, so showing too much is not an issue. */
+@media (any-hover: hover) {
+    .alectryon-bubble:hover:before,
+    .alectryon-toggle-label:hover:before,
+    .alectryon-io label.alectryon-input:hover:after {
+        background: #eeeeec;
+    }
+
+    .alectryon-io label.alectryon-input:hover {
+        text-decoration: underline dotted #babdb6;
+	    text-shadow: 0 0 1px rgb(46, 52, 54, 0.3); /* #2e3436 + opacity */
+    }
+
+    .alectryon-io .alectryon-sentence:hover .alectryon-output,
+    .alectryon-io .alectryon-token:hover .alectryon-type-info-wrapper,
+    .alectryon-io .alectryon-token:hover .alectryon-type-info-wrapper {
+        z-index: 2; /* Place hovered goals above .alectryon-sentence.alectryon-target ones */
+    }
+}
+
+.alectryon-toggle:checked + .alectryon-toggle-label:before,
+.alectryon-io .alectryon-sentence > .alectryon-toggle:checked + label.alectryon-input:after,
+.alectryon-io .alectryon-extra-goal-toggle:checked + .alectryon-goal > label:before {
+    background-color: #babdb6;
+    border-color: #babdb6;
+}
+
+/* Disable clicks on sentences when the document-wide toggle is set. */
+.alectryon-toggle:checked + label + .alectryon-container label.alectryon-input {
+    cursor: unset;
+    pointer-events: none;
+}
+
+/* Hide individual checkboxes when the document-wide toggle is set. */
+.alectryon-toggle:checked + label + .alectryon-container label.alectryon-input:after {
+    display: none;
+}
+
+/* .alectryon-output is displayed by toggles, :hover, and .alectryon-target rules */
+.alectryon-io .alectryon-output {
+    box-sizing: border-box;
+    display: none;
+    left: 0;
+    right: 0;
+    position: absolute;
+    padding: 0.25em 0;
+    overflow: visible; /* Let box-shadows overflow */
+    z-index: 1; /* Default to an index lower than that used by :hover */
+}
+
+.alectryon-io .alectryon-type-info-wrapper {
+    position: absolute;
+    display: inline-block;
+    width: 100%;
+}
+
+.alectryon-io .alectryon-type-info-wrapper.full-width {
+    left: 0;
+    min-width: 100%;
+    max-width: 100%;
+}
+
+.alectryon-io .alectryon-type-info .goal-separator {
+    height: unset;
+    margin-top: 0em;
+}
+
+.alectryon-io .alectryon-type-info-wrapper .alectryon-type-info {
+    box-sizing: border-box;
+    bottom: 100%;
+    position: absolute;
+    /*padding: 0.25em 0;*/
+    visibility: hidden;
+    overflow: visible; /* Let box-shadows overflow */
+    z-index: 1; /* Default to an index lower than that used by :hover */
+    white-space: pre-wrap !important;
+}
+
+.alectryon-io .alectryon-type-info-wrapper .alectryon-type-info .alectryon-goal.alectryon-docstring {
+    white-space: pre-wrap !important;
+}
+
+@media (any-hover: hover) { /* See note above about this @media query */
+    .alectryon-io .alectryon-sentence:hover .alectryon-output:not(:hover) {
+        display: block;
+    }
+
+    .alectryon-io.output-hidden .alectryon-sentence:hover .alectryon-output:not(:hover) {
+        display: none !important;
+    }
+
+    .alectryon-io.type-info-hidden .alectryon-token:hover .alectryon-type-info-wrapper .alectryon-type-info,
+    .alectryon-io.type-info-hidden .alectryon-token:hover .alectryon-type-info-wrapper .alectryon-type-info {
+        /*visibility: hidden !important;*/
+    }
+
+    .alectryon-io .alectryon-token:hover .alectryon-type-info-wrapper .alectryon-type-info,
+    .alectryon-io .alectryon-token:hover .alectryon-type-info-wrapper .alectryon-type-info {
+        visibility: visible;
+        transition-delay: 0.5s;
+    }
+}
+
+.alectryon-io .alectryon-sentence.alectryon-target .alectryon-output {
+    display: block;
+}
+
+/* Indicate active (hovered or targeted) goals with a shadow. */
+.alectryon-io .alectryon-sentence:hover .alectryon-output:not(:hover) .alectryon-messages,
+.alectryon-io .alectryon-sentence.alectryon-target .alectryon-output  .alectryon-messages,
+.alectryon-io .alectryon-sentence:hover .alectryon-output:not(:hover) .alectryon-goals,
+.alectryon-io .alectryon-sentence.alectryon-target .alectryon-output  .alectryon-goals,
+.alectryon-io .alectryon-token:hover .alectryon-type-info-wrapper .alectryon-type-info {
+    box-shadow: 2px 2px 2px gray;
+}
+
+.alectryon-io .alectryon-extra-goals .alectryon-goal .goal-hyps {
+    display: none;
+}
+
+.alectryon-io .alectryon-extra-goals .alectryon-extra-goal-toggle:not(:checked) + .alectryon-goal label.goal-separator hr {
+    /* Dashes indicate that the hypotheses are hidden */
+    border-top-style: dashed;
+}
+
+
+/* Show just a small preview of the other goals; this is undone by the
+   "extra-goal" toggle and by :hover and .alectryon-target in windowed mode. */
+.alectryon-io .alectryon-extra-goals .alectryon-goal .goal-conclusion {
+    max-height: 5.2em;
+    overflow-y: auto;
+    /* Combining ‘overflow-y: auto’ with ‘display: inline-block’ causes extra space
+       to be added below the box. ‘vertical-align: middle’ gets rid of it. */
+    vertical-align: middle;
+}
+
+.alectryon-io .alectryon-goals,
+.alectryon-io .alectryon-messages {
+    background: #f6f7f6;
+	/*border: thin solid #d3d7cf; /* Convenient when pre's background is already #EEE */
+    display: block;
+    padding: 0.25em;
+}
+
+.alectryon-message::before {
+    content: '';
+    float: right;
+    /* etc/svg/square-bubble-xl.svg */
+    background: url("data:image/svg+xml,%3Csvg width='14' height='14' viewBox='0 0 3.704 3.704' xmlns='http://www.w3.org/2000/svg'%3E%3Cg fill-rule='evenodd' stroke='%23000' stroke-width='.264'%3E%3Cpath d='M.794.934h2.115M.794 1.463h1.455M.794 1.992h1.852'/%3E%3C/g%3E%3Cpath d='M.132.14v2.646h.794v.661l.926-.661h1.72V.14z' fill='none' stroke='%23000' stroke-width='.265'/%3E%3C/svg%3E") top right no-repeat;
+    height: 14px;
+    width: 14px;
+}
+
+.alectryon-toggle:checked + label + .alectryon-container {
+    width: unset;
+}
+
+/* Show goals when a toggle is set */
+.alectryon-toggle:checked + label + .alectryon-container label.alectryon-input + .alectryon-output,
+.alectryon-io .alectryon-sentence > .alectryon-toggle:checked ~ .alectryon-output {
+    display: block;
+    position: static;
+    width: unset;
+    background: unset; /* Override the backgrounds set in floating in windowed mode */
+    padding: 0.25em 0; /* Re-assert so that later :hover rules don't override this padding */
+}
+
+.alectryon-toggle:checked + label + .alectryon-container label.alectryon-input + .alectryon-output .goal-hyps,
+.alectryon-io .alectryon-sentence > .alectryon-toggle:checked ~ .alectryon-output .goal-hyps {
+    /* Overridden back in windowed style */
+    flex-flow: row wrap;
+    justify-content: flex-start;
+}
+
+.alectryon-toggle:checked + label + .alectryon-container .alectryon-sentence .alectryon-output > div,
+.alectryon-io .alectryon-sentence > .alectryon-toggle:checked ~ .alectryon-output > div {
+    display: block;
+}
+
+.alectryon-io .alectryon-extra-goal-toggle:checked + .alectryon-goal .goal-hyps {
+    display: flex;
+}
+
+.alectryon-io .alectryon-extra-goal-toggle:checked + .alectryon-goal .goal-conclusion {
+    max-height: unset;
+    overflow-y: unset;
+}
+
+.alectryon-toggle:checked + label + .alectryon-container .alectryon-sentence > .alectryon-toggle ~ .alectryon-wsp,
+.alectryon-io .alectryon-sentence > .alectryon-toggle:checked ~ .alectryon-wsp {
+    display: none;
+}
+
+.alectryon-io .alectryon-messages,
+.alectryon-io .alectryon-message,
+.alectryon-io .alectryon-goals,
+.alectryon-io .alectryon-goal,
+.alectryon-io .goal-hyps > span,
+.alectryon-io .goal-conclusion {
+    border-radius: 0.15em;
+}
+
+.alectryon-io .alectryon-goal,
+.alectryon-io .alectryon-message {
+    align-items: center;
+    background: #f6f7f6;
+    border: 0em;
+    display: block;
+    flex-direction: column;
+    margin: 0.25em;
+    padding: 0.5em;
+    position: relative;
+}
+
+.alectryon-io .goal-hyps {
+    align-content: space-around;
+    align-items: baseline;
+    display: flex;
+    flex-flow: column nowrap; /* re-stated in windowed mode */
+    justify-content: space-around;
+    /* LATER use a ‘gap’ property instead of margins once supported */
+    margin: -0.15em -0.25em; /* -0.15em to cancel the item spacing */
+    padding-bottom: 0.35em; /* 0.5em-0.15em to cancel the 0.5em of .goal-separator */
+}
+
+.alectryon-io .goal-hyps > br {
+    display: none; /* Only for RSS readers */
+}
+
+.alectryon-io .goal-hyps > span,
+.alectryon-io .goal-conclusion {
+    /*background: #eeeeec;*/
+    display: inline-block;
+    padding: 0.15em 0.35em;
+}
+
+.alectryon-io .goal-hyps > span {
+    align-items: baseline;
+    display: inline-flex;
+    margin: 0.15em 0.25em;
+}
+
+.alectryon-block var,
+.alectryon-inline var,
+.alectryon-io .goal-hyps > span > var {
+    font-weight: 600;
+    font-style: unset;
+}
+
+.alectryon-io .goal-hyps > span > var {
+    /* Shrink the list of names, but let it grow as long as space is available. */
+    flex-basis: min-content;
+    flex-grow: 1;
+}
+
+.alectryon-io .goal-hyps > span b {
+    font-weight: 600;
+    margin: 0 0 0 0.5em;
+    white-space: pre;
+}
+
+.alectryon-io .hyp-body,
+.alectryon-io .hyp-type {
+    display: flex;
+    align-items: baseline;
+}
+
+.alectryon-io .goal-separator {
+    align-items: center;
+    display: flex;
+    flex-direction: row;
+    height: 1em; /* Fixed height to ignore goal name and markers */
+    margin-top: -0.5em; /* Compensated in .goal-hyps when shown */
+}
+
+.alectryon-io .goal-separator hr {
+    border: none;
+    border-top: thin solid #555753;
+    display: block;
+    flex-grow: 1;
+    margin: 0;
+}
+
+.alectryon-io .goal-separator .goal-name {
+    font-size: 0.75em;
+    margin-left: 0.5em;
+}
+
+/**********/
+/* Banner */
+/**********/
+
+.alectryon-banner {
+    background: #eeeeec;
+    border: 1px solid #babcbd;
+    font-size: 0.75em;
+    padding: 0.25em;
+    text-align: center;
+    margin: 1em 0;
+}
+
+.alectryon-banner a {
+    cursor: pointer;
+    text-decoration: underline;
+}
+
+.alectryon-banner kbd {
+    background: #d3d7cf;
+    border-radius: 0.15em;
+    border: 1px solid #babdb6;
+    box-sizing: border-box;
+    display: inline-block;
+    font-family: inherit;
+    font-size: 0.9em;
+    height: 1.3em;
+    line-height: 1.2em;
+    margin: -0.25em 0;
+    padding: 0 0.25em;
+    vertical-align: middle;
+}
+
+/**********/
+/* Toggle */
+/**********/
+
+.alectryon-toggle-label {
+    margin: 1rem 0;
+}
+
+/******************/
+/* Floating style */
+/******************/
+
+/* If there's space, display goals to the right of the code, not below it. */
+@media (min-width: 80rem) {
+    /* Unlike the windowed case, we don't want to move output blocks to the side
+       when they are both :checked and -targeted, since it gets confusing as
+       things jump around; hence the commented-output part of the selector,
+       which would otherwise increase specificity */
+    .alectryon-floating .alectryon-sentence.alectryon-target /* > .alectryon-toggle ~ */ .alectryon-output,
+    .alectryon-floating .alectryon-sentence:hover .alectryon-output {
+        top: 0;
+        left: 100%;
+        right: -100%;
+        padding: 0 0.5em;
+        position:  absolute;
+    }
+
+    .alectryon-floating .alectryon-output {
+        min-height: 100%;
+    }
+
+    .alectryon-floating .alectryon-sentence:hover .alectryon-output {
+        background: white; /* Ensure that short goals hide long ones */
+    }
+
+    /* This odd margin-bottom property prevents the sticky div from bumping
+       against the bottom of its container (.alectryon-output).  The alternative
+       would be enlarging .alectryon-output, but that would cause overflows,
+       enlarging scrollbars and yielding scrolling towards the bottom of the
+       page.  Doing things this way instead makes it possible to restrict
+       .alectryon-output to a reasonable size (100%, through top = bottom = 0).
+       See also https://stackoverflow.com/questions/43909940/. */
+    /* See note on specificity above */
+    .alectryon-floating .alectryon-sentence.alectryon-target /* > .alectryon-toggle ~ */ .alectryon-output > div,
+    .alectryon-floating .alectryon-sentence:hover .alectryon-output > div {
+        margin-bottom: -200%;
+        position: sticky;
+        top: 0;
+    }
+
+    .alectryon-floating .alectryon-toggle:checked + label + .alectryon-container .alectryon-sentence .alectryon-output > div,
+    .alectryon-floating .alectryon-io .alectryon-sentence > .alectryon-toggle:checked ~ .alectryon-output > div {
+        margin-bottom: unset; /* Undo the margin */
+    }
+
+    /* Float underneath the current fragment
+    @media (max-width: 80rem) {
+        .alectryon-floating .alectryon-output {
+            top: 100%;
+        }
+    } */
+}
+
+/********************/
+/* Multi-pane style */
+/********************/
+
+.alectryon-windowed {
+    border: 0 solid #2e3436;
+    box-sizing: border-box;
+}
+
+.alectryon-windowed .alectryon-sentence:hover .alectryon-output {
+    background: white; /* Ensure that short goals hide long ones */
+}
+
+.alectryon-windowed .alectryon-output {
+    position: fixed; /* Overwritten by the ‘:checked’ rules */
+}
+
+/* See note about specificity below */
+.alectryon-windowed .alectryon-sentence:hover .alectryon-output,
+.alectryon-windowed .alectryon-sentence.alectryon-target > .alectryon-toggle ~ .alectryon-output {
+    padding: 0.5em;
+    overflow-y: auto; /* Windowed contents may need to scroll */
+}
+
+.alectryon-windowed .alectryon-io .alectryon-sentence:hover .alectryon-output:not(:hover) .alectryon-messages,
+.alectryon-windowed .alectryon-io .alectryon-sentence.alectryon-target .alectryon-output  .alectryon-messages,
+.alectryon-windowed .alectryon-io .alectryon-sentence:hover .alectryon-output:not(:hover) .alectryon-goals,
+.alectryon-windowed .alectryon-io .alectryon-sentence.alectryon-target .alectryon-output  .alectryon-goals {
+    box-shadow: none; /* A shadow is unnecessary here and incompatible with overflow-y set to auto */
+}
+
+.alectryon-windowed .alectryon-io .alectryon-sentence.alectryon-target .alectryon-output .goal-hyps {
+    /* Restated to override the :checked style */
+    flex-flow: column nowrap;
+    justify-content: space-around;
+}
+
+
+.alectryon-windowed .alectryon-sentence.alectryon-target .alectryon-extra-goals .alectryon-goal .goal-conclusion
+/* Like .alectryon-io .alectryon-extra-goal-toggle:checked + .alectryon-goal .goal-conclusion */ {
+    max-height: unset;
+    overflow-y: unset;
+}
+
+.alectryon-windowed .alectryon-output > div {
+    display: flex; /* Put messages after goals */
+    flex-direction: column-reverse;
+}
+
+/*********************/
+/* Standalone styles */
+/*********************/
+
+.alectryon-standalone {
+    font-family: 'IBM Plex Serif', 'PT Serif', 'Merriweather', 'DejaVu Serif', serif;
+    line-height: 1.5;
+}
+
+@media screen and (min-width: 50rem) {
+    html.alectryon-standalone {
+        /* Prevent flickering when hovering a block causes scrollbars to appear. */
+        margin-left: calc(100vw - 100%);
+        margin-right: 0;
+    }
+}
+
+/* Coqdoc */
+
+.alectryon-coqdoc .doc .code,
+.alectryon-coqdoc .doc .inlinecode,
+.alectryon-coqdoc .doc .comment {
+    display: inline;
+}
+
+.alectryon-coqdoc .doc .comment {
+    color: #eeeeec;
+}
+
+.alectryon-coqdoc .doc .paragraph {
+    height: 0.75em;
+}
+
+/* Centered, Floating */
+
+.alectryon-standalone .alectryon-centered,
+.alectryon-standalone .alectryon-floating {
+    max-width: 50rem;
+    margin: auto;
+}
+
+@media (min-width: 80rem) {
+    .alectryon-standalone .alectryon-floating {
+        max-width: 80rem;
+    }
+
+    .alectryon-standalone .alectryon-floating > * {
+        width: 50%;
+        margin-left: 0;
+    }
+}
+
+/* Windowed */
+
+.alectryon-standalone .alectryon-windowed {
+    display: block;
+    margin: 0;
+    overflow-y: auto;
+    position: absolute;
+    padding: 0 1em;
+}
+
+.alectryon-standalone .alectryon-windowed > * {
+    /* Override properties of docutils_basic.css */
+    margin-left: 0;
+    max-width: unset;
+}
+
+.alectryon-standalone .alectryon-windowed .alectryon-io {
+    box-sizing: border-box;
+    width: 100%;
+}
+
+/* No need to predicate the ‘:hover’ rules below on ‘:not(:checked)’, since ‘left’,
+   ‘right’, ‘top’, and ‘bottom’ will be inactived by the :checked rules setting
+   ‘position’ to ‘static’ */
+
+
+/* Specificity: We want the output to stay inline when hovered while unfolded
+   (:checked), but we want it to move when it's targeted (i.e. when the user
+   is browsing goals one by one using the keyboard, in which case we want to
+   goals to appear in consistent locations).  The selectors below ensure
+   that :hover < :checked < -targeted in terms of specificity. */
+/* LATER: Reimplement this stuff with CSS variables */
+.alectryon-windowed .alectryon-sentence.alectryon-target > .alectryon-toggle ~ .alectryon-output {
+    position: fixed;
+}
+
+@media screen and (min-width: 60rem) {
+    .alectryon-standalone .alectryon-windowed {
+        border-right-width: thin;
+        bottom: 0;
+        left: 0;
+        right: 50%;
+        top: 0;
+    }
+
+    .alectryon-standalone .alectryon-windowed .alectryon-sentence:hover .alectryon-output,
+    .alectryon-standalone .alectryon-windowed .alectryon-sentence.alectryon-target .alectryon-output {
+        bottom: 0;
+        left: 50%;
+        right: 0;
+        top: 0;
+    }
+}
+
+@media screen and (max-width: 60rem) {
+    .alectryon-standalone .alectryon-windowed {
+        border-bottom-width: 1px;
+        bottom: 40%;
+        left: 0;
+        right: 0;
+        top: 0;
+    }
+
+    .alectryon-standalone .alectryon-windowed .alectryon-sentence:hover .alectryon-output,
+    .alectryon-standalone .alectryon-windowed .alectryon-sentence.alectryon-target .alectryon-output {
+        bottom: 0;
+        left: 0;
+        right: 0;
+        top: 60%;
+    }
+}

doc/alectryon.js

@@ -0,0 +1,190 @@
+var Alectryon;
+(function(Alectryon) {
+    (function (slideshow) {
+        function anchor(sentence) { return "#" + sentence.id; }
+
+        function current_sentence() { return slideshow.sentences[slideshow.pos]; }
+
+        function unhighlight() {
+            var sentence = current_sentence();
+            if (sentence) sentence.classList.remove("alectryon-target");
+            slideshow.pos = -1;
+        }
+
+        function highlight(sentence) {
+            sentence.classList.add("alectryon-target");
+        }
+
+        function scroll(sentence) {
+            // Put the top of the current fragment close to the top of the
+            // screen, but scroll it out of view if showing it requires pushing
+            // the sentence past half of the screen.  If sentence is already in
+            // a reasonable position, don't move.
+            var parent = sentence.parentElement;
+            /* We want to scroll the whole document, so start at root… */
+            while (parent && !parent.classList.contains("alectryon-root"))
+                parent = parent.parentElement;
+            /* … and work up from there to find a scrollable element.
+               parent.scrollHeight can be greater than parent.clientHeight
+               without showing scrollbars, so we add a 10px buffer. */
+            while (parent && parent.scrollHeight <= parent.clientHeight + 10)
+                parent = parent.parentElement;
+            /* <body> and <html> elements can have their client rect overflow
+             * the window if their height is unset, so scroll the window
+             * instead */
+            if (parent && (parent.nodeName == "BODY" || parent.nodeName == "HTML"))
+                parent = null;
+
+            var rect = function(e) { return e.getBoundingClientRect(); };
+            var parent_box = parent ? rect(parent) : { y: 0, height: window.innerHeight },
+                sentence_y = rect(sentence).y - parent_box.y,
+                fragment_y = rect(sentence.parentElement).y - parent_box.y;
+
+            // The assertion below sometimes fails for the first element in a block.
+            // console.assert(sentence_y >= fragment_y);
+
+            if (sentence_y < 0.1 * parent_box.height ||
+                sentence_y > 0.7 * parent_box.height) {
+                (parent || window).scrollBy(
+                    0, Math.max(sentence_y - 0.5 * parent_box.height,
+                                fragment_y - 0.1 * parent_box.height));
+            }
+        }
+
+        function highlighted(pos) {
+            return slideshow.pos == pos;
+        }
+
+        function navigate(pos, inhibitScroll) {
+            unhighlight();
+            slideshow.pos = Math.min(Math.max(pos, 0), slideshow.sentences.length - 1);
+            var sentence = current_sentence();
+            highlight(sentence);
+            if (!inhibitScroll)
+                scroll(sentence);
+        }
+
+        var keys = {
+            PAGE_UP: 33,
+            PAGE_DOWN: 34,
+            ARROW_UP: 38,
+            ARROW_DOWN: 40,
+            h: 72, l: 76, p: 80, n: 78
+        };
+
+        function onkeydown(e) {
+            e = e || window.event;
+            if (e.ctrlKey || e.metaKey) {
+                if (e.keyCode == keys.ARROW_UP)
+                    slideshow.previous();
+                else if (e.keyCode == keys.ARROW_DOWN)
+                    slideshow.next();
+                else
+                    return;
+            } else {
+                // if (e.keyCode == keys.PAGE_UP || e.keyCode == keys.p || e.keyCode == keys.h)
+                //     slideshow.previous();
+                // else if (e.keyCode == keys.PAGE_DOWN || e.keyCode == keys.n || e.keyCode == keys.l)
+                //     slideshow.next();
+                // else
+                return;
+            }
+            e.preventDefault();
+        }
+
+        function start() {
+            slideshow.navigate(0);
+        }
+
+        function toggleHighlight(idx) {
+            if (highlighted(idx))
+                unhighlight();
+            else
+                navigate(idx, true);
+        }
+
+        function handleClick(evt) {
+            if (evt.ctrlKey || evt.metaKey) {
+                var sentence = evt.currentTarget;
+
+                // Ensure that the goal is shown on the side, not inline
+                var checkbox = sentence.getElementsByClassName("alectryon-toggle")[0];
+                if (checkbox)
+                    checkbox.checked = false;
+
+                toggleHighlight(sentence.alectryon_index);
+                evt.preventDefault();
+            }
+        }
+
+        function init() {
+            document.onkeydown = onkeydown;
+            slideshow.pos = -1;
+            slideshow.sentences = Array.from(document.getElementsByClassName("alectryon-sentence"));
+            slideshow.sentences.forEach(function (s, idx) {
+                s.addEventListener('click', handleClick, false);
+                s.alectryon_index = idx;
+            });
+        }
+
+        slideshow.start = start;
+        slideshow.end = unhighlight;
+        slideshow.navigate = navigate;
+        slideshow.next = function() { navigate(slideshow.pos + 1); };
+        slideshow.previous = function() { navigate(slideshow.pos + -1); };
+        window.addEventListener('DOMContentLoaded', init);
+    })(Alectryon.slideshow || (Alectryon.slideshow = {}));
+
+    (function (styles) {
+        var styleNames = ["centered", "floating", "windowed"];
+
+        function className(style) {
+            return "alectryon-" + style;
+        }
+
+        function setStyle(style) {
+            var root = document.getElementsByClassName("alectryon-root")[0];
+            styleNames.forEach(function (s) {
+                root.classList.remove(className(s)); });
+            root.classList.add(className(style));
+        }
+
+        function init() {
+            var banner = document.getElementsByClassName("alectryon-banner")[0];
+            if (banner) {
+                banner.append(" Style: ");
+                styleNames.forEach(function (styleName, idx) {
+                    var s = styleName;
+                    var a = document.createElement("a");
+                    a.onclick = function() { setStyle(s); };
+                    a.append(styleName);
+                    if (idx > 0) banner.append("; ");
+                    banner.appendChild(a);
+                });
+                banner.append(".");
+            }
+        }
+
+        window.addEventListener('DOMContentLoaded', init);
+
+        styles.setStyle = setStyle;
+    })(Alectryon.styles || (Alectryon.styles = {}));
+})(Alectryon || (Alectryon = {}));
+
+function setHidden(elements, isVisible, token) {
+    for (let i = 0; i < elements.length; i++) {
+        if (isVisible) {
+            elements[i].classList.remove(token)
+        } else {
+            elements[i].classList.add(token)
+        }
+    }
+}
+
+function toggleShowTypes(checkbox) {
+    setHidden(document.getElementsByClassName("alectryon-io"), checkbox.checked, "type-info-hidden")
+}
+
+function toggleShowGoals(checkbox) {
+    setHidden(document.getElementsByClassName("alectryon-io"), checkbox.checked, "output-hidden")
+}

doc/array.md

@@ -0,0 +1,77 @@
+# Arrays
+
+The `Array` type implements a *dynamic* (aka growable) array.
+It is defined as
+```lean
+# namespace hidden
+structure Array (α : Type u) where
+  data : List α
+# end hidden
+```
+but its execution time representation is optimized, and it is similar to C++ `std::vector<T>` and Rust `Vec<T>`.
+The Lean type checker has no special support for reducing `Array`s.
+
+You can create arrays in several ways. You can create a small array by listing consecutive values between
+`#[` and `]` and separated by commas, as shown in the following examples.
+
+```lean
+#check #[1, 2, 3] -- Array Nat
+
+#check #[] -- Array ?m
+```
+
+The type of the array elements is inferred from the literals used and must be consistent.
+```lean
+#check #["hello", "world"] -- Array String
+
+-- The following is not valid
+#check_failure #[10, "hello"]
+```
+Recall that the command `#check_failure <term>` only succeeds when the given term is not type correct.
+
+To create an array of size `n` in which all the elements are initialized to some value `a`, use `mkArray`.
+```lean
+#eval mkArray 5 'a'
+-- #['a', 'a', 'a', 'a', 'a']
+```
+
+## Accessing elements
+
+You can access array elements by using brackets (`[` and `]`).
+```lean
+def f (a : Array Nat) (i : Fin a.size) :=
+  a[i] + a[i]
+```
+Note that the index `i` has type `Fin a.size`, i.e., it is natural number less than `a.size`.
+You can also write
+```lean
+def f (a : Array Nat) (i : Nat) (h  : i < a.size) :=
+  a[i] + a[i]
+```
+The bracket operator is whitespace sensitive.
+
+```lean
+def f (xs : List Nat) : List Nat :=
+  xs ++ xs
+
+def as : Array Nat :=
+  #[1, 2, 3, 4]
+
+def idx : Fin 4 :=
+  2
+
+#eval f [1, 2, 3] -- This is a function application
+
+#eval as[idx] -- This is an array access
+```
+The notation `a[i]` has two variants: `a[i]!` and `a[i]?`. In both cases, `i` has type `Nat`. The first one
+produces a panic error message if the index `i` is out of bounds. The latter returns an `Option` type.
+
+```lean
+#eval #['a', 'b', 'c'][1]?
+-- some 'b'
+#eval #['a', 'b', 'c'][5]?
+-- none
+#eval #['a', 'b', 'c'][1]!
+-- 'b!
+```

doc/autobound.md

@@ -0,0 +1,47 @@
+## Auto Bound Implicit Arguments
+
+In the previous section, we have shown how implicit arguments make functions more convenient to use.
+However, functions such as `compose` are still quite verbose to define. Note that the universe
+polymorphic `compose` is even more verbose than the one previously defined.
+
+```lean
+universe u v w
+def compose {α : Type u} {β : Type v} {γ : Type w}
+            (g : β → γ) (f : α → β) (x : α) : γ :=
+  g (f x)
+```
+
+You can avoid the `universe` command by providing the universe parameters when defining `compose`.
+
+```lean
+def compose.{u, v, w}
+            {α : Type u} {β : Type v} {γ : Type w}
+            (g : β → γ) (f : α → β) (x : α) : γ :=
+  g (f x)
+```
+
+Lean 4 supports a new feature called *auto bound implicit arguments*. It makes functions such as
+`compose` much more convenient to write. When Lean processes the header of a declaration,
+any unbound identifier is automatically added as an implicit argument *if* it is a single lower case or
+greek letter. With this feature, we can write `compose` as
+
+```lean
+def compose (g : β → γ) (f : α → β) (x : α) : γ :=
+  g (f x)
+
+#check @compose
+-- {β : Sort u_1} → {γ : Sort u_2} → {α : Sort u_3} → (β → γ) → (α → β) → α → γ
+```
+Note that, Lean inferred a more general type using `Sort` instead of `Type`.
+
+Although we love this feature and use it extensively when implementing Lean,
+we realize some users may feel uncomfortable with it. Thus, you can disable it using
+the command `set_option autoImplicit false`.
+```lean
+set_option autoImplicit false
+/- The following definition produces `unknown identifier` errors -/
+-- def compose (g : β → γ) (f : α → β) (x : α) : γ :=
+--   g (f x)
+```
+The Lean language server provides [semantic highlighting](./semantic_highlighting.md) information to editors, and it provides
+visual feedback whether an identifier has been interpreted as an auto bound implicit argument.

doc/bin/README.md

@@ -0,0 +1,14 @@
+Lean binary distribution
+------------------------
+
+The binary distribution package contains:
+
+- Lean executable (located in the sub-directory bin)
+- Standard library (located in the sub-directory lib/lean/library)
+
+Assuming you are in the same directory this file is located,
+the following command executes a simple set of examples
+
+% bin/lean examples/ex.lean
+
+For more information on Lean and supported editors, please see https://lean-lang.org/documentation/.

doc/book.toml

@@ -0,0 +1,21 @@
+[book]
+authors = ["Leonardo de Moura", "Sebastian Ullrich"]
+language = "en"
+multilingual = false
+src = "."
+title = "Lean Manual"
+
+[build]
+build-dir = "out"
+
+[output.html]
+git-repository-url = "https://github.com/leanprover/lean4"
+additional-css = ["alectryon.css", "pygments.css"]
+additional-js = ["alectryon.js"]
+
+[output.html.fold]
+enable = true
+level = 0
+
+[output.html.playground.boring-prefixes]
+lean = "# "

doc/bool.md

@@ -0,0 +1 @@
+# Booleans

doc/builtintypes.md

@@ -0,0 +1,25 @@
+# Builtin Types
+
+## Numeric Operations
+
+Lean supports the basic mathematical operations you’d expect for all of the number types: addition, subtraction, multiplication, division, and remainder.
+The following code shows how you’d use each one in a `def` commands:
+
+```lean
+-- addition
+def sum := 5 + 10
+
+-- subtraction
+def difference := 95.5 - 4.3
+
+-- multiplication
+def product := 4 * 30
+
+-- division
+def quotient := 53.7 / 32.2
+
+-- remainder/modulo
+def modulo := 43 % 5
+```
+
+Each expression in these statements uses a mathematical operator and evaluates to a single value.

doc/char.md

@@ -0,0 +1,11 @@
+# Characters
+
+A value of type `Char`, also known as a character, is a [Unicode scalar value](https://www.unicode.org/glossary/#unicode_scalar_value). It is represented using an unsigned 32-bit integer and is statically guaranteed to be a valid Unicode scalar value.
+
+Syntactically, character literals are enclosed in single quotes.
+```lean
+#eval 'a' -- 'a'
+#eval '∀' -- '∀'
+```
+
+Characters are ordered and can be decidably compared using the relational operators `=`, `<`, `≤`, `>`, `≥`.

doc/declarations.md

@@ -0,0 +1,884 @@
+# Declarations
+
+-- TODO (fix)
+
+Declaration Names
+=================
+
+A declaration name is a hierarchical [identifier](lexical_structure.md#identifiers) that is interpreted relative to the current namespace as well as (during lookup) to the set of open namespaces.
+
+```lean
+namespace A
+  opaque B.c : Nat
+  #print B.c -- opaque A.B.c : Nat
+end A
+
+#print A.B.c -- opaque A.B.c : Nat
+open A
+#print B.c -- opaque A.B.c : Nat
+```
+
+Declaration names starting with an underscore are reserved for internal use. Names starting with the special atomic name ``_root_`` are interpreted as absolute names.
+
+```lean
+opaque a : Nat
+namespace A
+  opaque a : Int
+  #print _root_.a -- opaque a : Nat
+  #print A.a      -- opaque A.a : Int
+end A
+```
+
+Contexts and Telescopes
+=======================
+
+When processing user input, Lean first parses text to a raw expression format. It then uses background information and type constants to disambiguate overloaded symbols and infer implicit arguments, resulting in a fully-formed expression. This process is known as *elaboration*.
+
+As hinted in [Expression Syntax](expressions.md#expression_syntax),
+expressions are parsed and elaborated with respect to an *environment*
+and a *local context*. Roughly speaking, an environment represents the
+state of Lean at the point where an expression is parsed, including
+previously declared axioms, constants, definitions, and theorems. In a
+given environment, a *local context* consists of a sequence ``(a₁ :
+α₁) (a₂ : α₂) ... (aₙ : αₙ)`` where each ``aᵢ`` is a name denoting a
+local constant and each ``αᵢ`` is an expression of type ``Sort u`` for
+some ``u`` which can involve elements of the environment and the local
+constants ``aⱼ`` for ``j < i``.
+
+Intuitively, a local context is a list of variables that are held constant while an expression is being elaborated. Consider the following
+
+```lean
+def f (a b : Nat) : Nat → Nat := fun c => a + (b + c)
+```
+
+Here the expression ``fun c => a + (b + c)`` is elaborated in the context ``(a : Nat) (b : Nat)`` and the expression ``a + (b + c)`` is elaborated in the context ``(a : Nat) (b : Nat) (c : Nat)``. If you replace the expression ``a + (b + c)`` with an underscore, the error message from Lean will include the current *goal*:
+
+```
+a b c : Nat
+⊢ Nat
+```
+
+Here ``a b c : Nat`` indicates the local context, and the second ``Nat`` indicates the expected type of the result.
+
+A *context* is sometimes called a *telescope*, but the latter is used more generally to include a sequence of declarations occurring relative to a given context. For example, relative to the context ``(a₁ : α₁) (a₂ : α₂) ... (aₙ : αₙ)``, the types ``βᵢ`` in a telescope ``(b₁ : β₁) (b₂ : β₂) ... (bₙ : βₙ)`` can refer to ``a₁, ..., aₙ``. Thus a context can be viewed as a telescope relative to the empty context.
+
+Telescopes are often used to describe a list of arguments, or parameters, to a declaration. In such cases, it is often notationally convenient to let ``(a : α)`` stand for a telescope rather than just a single argument. In general, the annotations described in [Implicit Arguments](expressions.md#implicit_arguments) can be used to mark arguments as implicit.
+
+.. _basic_declarations:
+
+Basic Declarations
+==================
+
+Lean provides ways of adding new objects to the environment. The following provide straightforward ways of declaring new objects:
+
+* ``axiom c : α`` :  declare a constant named ``c`` of type ``α``, it is postulating that `α` is not an empty type.
+* ``def c : α := v`` : defines ``c`` to denote ``v``, which should have type ``α``.
+* ``theorem c : p := v`` : similar to ``def``, but intended to be used when ``p`` is a proposition.
+* ``opaque c : α (:= v)?`` : declares a opaque constant named ``c`` of type ``α``, the optional value `v` is must have type `α`
+  and can be viewed as a certificate that ``α`` is not an empty type. If the value is not provided, Lean tries to find one
+  using a procedure based on type class resolution. The value `v` is hidden from the type checker. You can assume that
+  Lean "forgets" `v` after type checking this kind of declaration.
+
+It is sometimes useful to be able to simulate a definition or theorem without naming it or adding it to the environment.
+
+* ``example : α := t`` : elaborates ``t`` and checks that it has sort ``α`` (often a proposition), without adding it to the environment.
+
+In ``def``, the type (``α`` or ``p``, respectively) can be omitted when it can be inferred by Lean. Constants declared with ``theorem`` are marked as ``irreducible``.
+
+Any of ``def``, ``theorem``, ``axiom``, or ``example`` can take a list of arguments (that is, a context) before the colon. If ``(a : α)`` is a context, the definition ``def foo (a : α) : β := t``
+is interpreted as ``def foo : (a : α) → β := fun a : α => t``. Similarly, a theorem ``theorem foo (a : α) : p := t`` is interpreted as ``theorem foo : ∀ a : α, p := fun a : α => t``.
+
+```lean
+opaque c : Nat
+opaque d : Nat
+axiom cd_eq : c = d
+
+def foo : Nat := 5
+def bar := 6
+def baz (x y : Nat) (s : List Nat) := [x, y] ++ s
+
+theorem foo_eq_five : foo = 5 := rfl
+theorem baz_theorem (x y : Nat) : baz x y [] = [x, y] := rfl
+
+example (x y : Nat) : baz x y [] = [x, y] := rfl
+```
+
+Inductive Types
+===============
+
+Lean's axiomatic foundation allows users to declare arbitrary
+inductive families, following the pattern described by [Dybjer]_. To
+make the presentation more manageable, we first describe inductive
+*types*, and then describe the generalization to inductive *families*
+in the next section. The declaration of an inductive type has the
+following form:
+
+```
+inductive Foo (a : α) where
+  | constructor₁ : (b : β₁) → Foo a
+  | constructor₂ : (b : β₂) → Foo a
+  ...
+  | constructorₙ : (b : βₙ) → Foo a
+```
+
+Here ``(a : α)`` is a context and each ``(b : βᵢ)`` is a telescope in the context ``(a : α)`` together with ``Foo``, subject to the following constraints.
+
+Suppose the telescope ``(b : βᵢ)`` is ``(b₁ : βᵢ₁) ... (bᵤ : βᵢᵤ)``. Each argument in the telescope is either *nonrecursive* or *recursive*.
+
+- An argument ``(bⱼ : βᵢⱼ)`` is *nonrecursive* if ``βᵢⱼ`` does not refer to ``foo,`` the inductive type being defined. In that case, ``βᵢⱼ`` can be any type, so long as it does not refer to any nonrecursive arguments.
+
+- An argument ``(bⱼ : βᵢⱼ)`` is *recursive* if it ``βᵢⱼ`` of the form ``Π (d : δ), foo`` where ``(d : δ)`` is a telescope which does not refer to ``foo`` or any nonrecursive arguments.
+
+The inductive type ``foo`` represents a type that is freely generated by the constructors. Each constructor can take arbitrary data and facts as arguments (the nonrecursive arguments), as well as indexed sequences of elements of ``foo`` that have been previously constructed (the recursive arguments). In set theoretic models, such sets can be represented by well-founded trees labeled by the constructor data, or they can defined using other transfinite or impredicative means.
+
+The declaration of the type ``foo`` as above results in the addition of the following constants to the environment:
+
+- the *type former* ``foo : Π (a : α), Sort u``
+- for each ``i``, the *constructor* ``foo.constructorᵢ : Π (a : α) (b : βᵢ), foo a``
+- the *eliminator* ``foo.rec``, which takes arguments
+
+  + ``(a : α)`` (the parameters)
+  + ``{C : foo a → Type u}`` (the *motive* of the elimination)
+  + for each ``i``, the *minor premise* corresponding to ``constructorᵢ``
+  + ``(x : foo)`` (the *major premise*)
+
+  and returns an element of ``C x``. Here, The ith minor premise is a function which takes
+
+  +  ``(b : βᵢ)`` (the arguments to the constructor)
+  + an argument of type ``Π (d : δ), C (bⱼ d)`` corresponding to each recursive argument ``(bⱼ : βᵢⱼ)``, where ``βᵢⱼ``  is of the form ``Π (d : δ), foo`` (the recursive values of the function being defined)
+
+  and returns an element of ``C (constructorᵢ a b)``, the intended value of the function at ``constructorᵢ a b``.
+
+The eliminator represents a principle of recursion: to construct an element of ``C x`` where ``x : foo a``, it suffices to consider each of the cases where ``x`` is of the form ``constructorᵢ a b`` and to provide an auxiliary construction in each case. In the case where some of the arguments to ``constructorᵢ`` are recursive, we can assume that we have already constructed values of ``C y`` for each value ``y`` constructed at an earlier stage.
+
+Under the propositions-as-type correspondence, when ``C x`` is an element of ``Prop``, the eliminator represents a principle of induction. In order to show ``∀ x, C x``, it suffices to show that ``C`` holds for each constructor, under the inductive hypothesis that it holds for all recursive inputs to the constructor.
+
+The eliminator and constructors satisfy the following identities, in which all the arguments are shown explicitly. Suppose we set ``F := foo.rec a C f₁ ... fₙ``. Then for each constructor, we have the definitional reduction:
+
+```
+F (constructorᵢ a b) = fᵢ b ... (fun d : δᵢⱼ => F (bⱼ d)) ...
+```
+
+where the ellipses include one entry for each recursive argument.
+
+Below are some common examples of inductive types, many of which are defined in the core library.
+
+```lean
+namespace Hide
+universe u v
+
+-- BEGIN
+inductive Empty : Type
+
+inductive Unit : Type
+| unit : Unit
+
+inductive Bool : Type
+| false : Bool
+| true : Bool
+
+inductive Prod (α : Type u) (β : Type v) : Type (max u v)
+| mk : α → β → Prod α β
+
+inductive Sum (α : Type u) (β : Type v)
+| inl : α → Sum α β
+| inr : β → Sum α β
+
+inductive Sigma (α : Type u) (β : α → Type v)
+| mk : (a : α) → β a → Sigma α β
+
+inductive false : Prop
+
+inductive True : Prop
+| trivial : True
+
+inductive And (p q : Prop) : Prop
+| intro : p → q → And p q
+
+inductive Or (p q : Prop) : Prop
+| inl : p → Or p q
+| inr : q → Or p q
+
+inductive Exists (α : Type u) (p : α → Prop) : Prop
+| intro : ∀ x : α, p x → Exists α p
+
+inductive Subtype (α : Type u) (p : α → Prop) : Type u
+| intro : ∀ x : α, p x → Subtype α p
+
+inductive Nat : Type
+| zero : Nat
+| succ : Nat → Nat
+
+inductive List (α : Type u)
+| nil : List α
+| cons : α → List α → List α
+
+-- full binary tree with nodes and leaves labeled from α
+inductive BinTree (α : Type u)
+| leaf : α → BinTree α
+| node : BinTree α → α → BinTree α → BinTree α
+
+-- every internal node has subtrees indexed by Nat
+inductive CBT (α : Type u)
+| leaf : α → CBT α
+| node : (Nat → CBT α) → CBT α
+-- END
+end Hide
+```
+
+Note that in the syntax of the inductive definition ``Foo``, the context ``(a : α)`` is left implicit. In other words, constructors and recursive arguments are written as though they have return type ``Foo`` rather than ``Foo a``.
+
+Elements of the context ``(a : α)`` can be marked implicit as described in [Implicit Arguments](#implicit.md#implicit_arguments). These annotations bear only on the type former, ``Foo``. Lean uses a heuristic to determine which arguments to the constructors should be marked implicit, namely, an argument is marked implicit if it can be inferred from the type of a subsequent argument. If the annotation ``{}`` appears after the constructor, a argument is marked implicit if it can be inferred from the type of a subsequent argument *or the return type*. For example, it is useful to let ``nil`` denote the empty list of any type, since the type can usually be inferred in the context in which it appears. These heuristics are imperfect, and you may sometimes wish to define your own constructors in terms of the default ones. In that case, use the ``[match_pattern]`` [attribute](TODO: missing link) to ensure that these will be used appropriately by the [Equation Compiler](#the-equation-compiler).
+
+There are restrictions on the universe ``u`` in the return type ``Sort u`` of the type former. There are also restrictions on the universe ``u`` in the return type ``Sort u`` of the motive of the eliminator. These will be discussed in the next section in the more general setting of inductive families.
+
+Lean allows some additional syntactic conveniences. You can omit the return type of the type former, ``Sort u``, in which case Lean will infer the minimal possible nonzero value for ``u``. As with function definitions, you can list arguments to the constructors before the colon. In an enumerated type (that is, one where the constructors have no arguments), you can also leave out the return type of the constructors.
+
+```lean
+namespace Hide
+universe u
+
+-- BEGIN
+inductive Weekday
+| sunday | monday | tuesday | wednesday
+| thursday | friday | saturday
+
+inductive Nat
+| zero
+| succ (n : Nat) : Nat
+
+inductive List (α : Type u)
+| nil : List α
+| cons (a : α) (l : List α) : List α
+
+@[match_pattern]
+def List.nil' (α : Type u) : List α := List.nil
+
+def length {α : Type u} : List α → Nat
+| (List.nil' _) => 0
+| (List.cons a l) => 1 + length l
+-- END
+
+end Hide
+```
+
+The type former, constructors, and eliminator are all part of Lean's axiomatic foundation, which is to say, they are part of the trusted kernel. In addition to these axiomatically declared constants, Lean automatically defines some additional objects in terms of these, and adds them to the environment. These include the following:
+
+- ``Foo.recOn`` : a variant of the eliminator, in which the major premise comes first
+- ``Foo.casesOn`` : a restricted version of the eliminator which omits any recursive calls
+- ``Foo.noConfusionType``, ``Foo.noConfusion`` : functions which witness the fact that the inductive type is freely generated, i.e. that the constructors are injective and that distinct constructors produce distinct objects
+- ``Foo.below``, ``Foo.ibelow`` : functions used by the equation compiler to implement structural recursion
+- ``instance : SizeOf Foo`` : a measure which can be used for well-founded recursion
+
+Note that it is common to put definitions and theorems related to a datatype ``foo`` in a namespace of the same name. This makes it possible to use projection notation described in [Structures](struct.md#structures) and [Namespaces](namespaces.md#namespaces).
+
+```lean
+namespace Hide
+universe u
+
+-- BEGIN
+inductive Nat
+| zero
+| succ (n : Nat) : Nat
+
+#check Nat
+#check @Nat.rec
+#check Nat.zero
+#check Nat.succ
+
+#check @Nat.recOn
+#check @Nat.casesOn
+#check @Nat.noConfusionType
+#check @Nat.noConfusion
+#check @Nat.brecOn
+#check Nat.below
+#check Nat.ibelow
+#check Nat._sizeOf_1
+
+-- END
+
+end Hide
+```
+
+.. _inductive_families:
+
+Inductive Families
+==================
+
+In fact, Lean implements a slight generalization of the inductive types described in the previous section, namely, inductive *families*. The declaration of an inductive family in Lean has the following form:
+
+```
+inductive Foo (a : α) : Π (c : γ), Sort u
+| constructor₁ : Π (b : β₁), Foo t₁
+| constructor₂ : Π (b : β₂), Foo t₂
+...
+| constructorₙ : Π (b : βₙ), Foo tₙ
+```
+
+Here ``(a : α)`` is a context, ``(c : γ)`` is a telescope in context ``(a : α)``, each ``(b : βᵢ)`` is a telescope in the context ``(a : α)`` together with ``(Foo : Π (c : γ), Sort u)`` subject to the constraints below, and each ``tᵢ`` is a tuple of terms in the context ``(a : α) (b : βᵢ)`` having the types ``γ``. Instead of defining a single inductive type ``Foo a``, we are now defining a family of types ``Foo a c`` indexed by elements ``c : γ``.  Each constructor, ``constructorᵢ``, places its result in the type ``Foo a tᵢ``, the member of the family with index ``tᵢ``.
+
+The modifications to the scheme in the previous section are straightforward. Suppose the telescope ``(b : βᵢ)`` is ``(b₁ : βᵢ₁) ... (bᵤ : βᵢᵤ)``.
+
+- As before, an argument ``(bⱼ : βᵢⱼ)`` is *nonrecursive* if ``βᵢⱼ`` does not refer to ``Foo,`` the inductive type being defined. In that case, ``βᵢⱼ`` can be any type, so long as it does not refer to any nonrecursive arguments.
+
+- An argument ``(bⱼ : βᵢⱼ)`` is *recursive* if ``βᵢⱼ`` is of the form ``Π (d : δ), Foo s`` where ``(d : δ)`` is a telescope which does not refer to ``Foo`` or any nonrecursive arguments and ``s`` is a tuple of terms in context ``(a : α)`` and the previous nonrecursive ``bⱼ``'s with types ``γ``.
+
+The declaration of the type ``Foo`` as above results in the addition of the following constants to the environment:
+
+- the *type former* ``Foo : Π (a : α) (c : γ), Sort u``
+- for each ``i``, the *constructor* ``Foo.constructorᵢ : Π (a : α) (b : βᵢ), Foo a tᵢ``
+- the *eliminator* ``Foo.rec``, which takes arguments
+
+  + ``(a : α)`` (the parameters)
+  + ``{C : Π (c : γ), Foo a c → Type u}`` (the motive of the elimination)
+  + for each ``i``, the minor premise corresponding to ``constructorᵢ``
+  + ``(x : Foo a)`` (the major premise)
+
+  and returns an element of ``C x``. Here, The ith minor premise is a function which takes
+
+  +  ``(b : βᵢ)`` (the arguments to the constructor)
+  + an argument of type ``Π (d : δ), C s (bⱼ d)`` corresponding to each recursive argument ``(bⱼ : βᵢⱼ)``, where ``βᵢⱼ``  is of the form ``Π (d : δ), Foo s``
+
+  and returns an element of ``C tᵢ (constructorᵢ a b)``.
+
+Suppose we set ``F := Foo.rec a C f₁ ... fₙ``. Then for each constructor, we have the definitional reduction, as before:
+
+```
+F (constructorᵢ a b) = fᵢ b ... (fun d : δᵢⱼ => F (bⱼ d)) ...
+```
+
+where the ellipses include one entry for each recursive argument.
+
+The following are examples of inductive families.
+
+```lean
+namespace Hide
+universe u
+
+-- BEGIN
+inductive Vector (α : Type u) : Nat → Type u
+| nil  : Vector 0
+| succ : Π n, Vector n → Vector (n + 1)
+
+-- 'IsProd s n' means n is a product of elements of s
+inductive IsProd (s : Set Nat) : Nat → Prop
+| base : ∀ n ∈ s, IsProd n
+| step : ∀ m n, IsProd m → IsProd n → IsProd (m * n)
+
+inductive Eq {α : Sort u} (a : α) : α → Prop
+| refl : Eq a
+-- END
+
+end Hide
+```
+
+We can now describe the constraints on the return type of the type former, ``Sort u``. We can always take ``u`` to be ``0``, in which case we are defining an inductive family of propositions. If ``u`` is nonzero, however, it must satisfy the following constraint: for each type ``βᵢⱼ : Sort v`` occurring in the constructors, we must have ``u ≥ v``. In the set-theoretic interpretation, this ensures that the universe in which the resulting type resides is large enough to contain the inductively generated family, given the number of distinctly-labeled constructors. The restriction does not hold for inductively defined propositions, since these contain no data.
+
+Putting an inductive family in ``Prop``, however, does impose a restriction on the eliminator. Generally speaking, for an inductive family in ``Prop``, the motive in the eliminator is required to be in ``Prop``. But there is an exception to this rule: you are allowed to eliminate from an inductively defined ``Prop`` to an arbitrary ``Sort`` when there is only one constructor, and each argument to that constructor is either in ``Prop`` or an index. The intuition is that in this case the elimination does not make use of any information that is not already given by the mere fact that the type of argument is inhabited. This special case is known as *singleton elimination*.
+
+.. _mutual_and_nested_inductive_definitions:
+
+Mutual and Nested Inductive Definitions
+=======================================
+
+Lean supports two generalizations of the inductive families described above, namely, *mutual* and *nested* inductive definitions. These are *not* implemented natively in the kernel. Rather, the definitions are compiled down to the primitive inductive types and families.
+
+The first generalization allows for multiple inductive types to be defined simultaneously.
+
+```
+mutual
+
+inductive Foo (a : α) : Π (c : γ₁), Sort u
+| constructor₁₁ : Π (b : β₁₁), Foo a t₁₁
+| constructor₁₂ : Π (b : β₁₂), Foo a t₁₂
+...
+| constructor₁ₙ : Π (b : β₁ₙ), Foo a t₁ₙ
+
+inductive Bar (a : α) : Π (c : γ₂), Sort u
+| constructor₂₁ : Π (b : β₂₁), Bar a t₂₁
+| constructor₂₂ : Π (b : β₂₂), Bar a t₂₂
+...
+| constructor₂ₘ : Π (b : β₂ₘ), Bar a t₂ₘ
+
+end
+```
+
+Here the syntax is shown for defining two inductive families, ``Foo`` and ``Bar``, but any number is allowed. The restrictions are almost the same as for ordinary inductive families. For example, each ``(b : βᵢⱼ)`` is a telescope relative to the context ``(a : α)``. The difference is that the constructors can now have recursive arguments whose return types are any of the inductive families currently being defined, in this case ``Foo`` and ``Bar``. Note that all of the inductive definitions share the same parameters ``(a : α)``, though they may have different indices.
+
+A mutual inductive definition is compiled down to an ordinary inductive definition using an extra finite-valued index to distinguish the components. The details of the internal construction are meant to be hidden from most users. Lean defines the expected type formers ``Foo`` and ``Bar`` and constructors ``constructorᵢⱼ`` from the internal inductive definition. There is no straightforward elimination principle, however. Instead, Lean defines an appropriate ``sizeOf`` measure, meant for use with well-founded recursion, with the property that the recursive arguments to a constructor are smaller than the constructed value.
+
+The second generalization relaxes the restriction that in the recursive definition of ``Foo``, ``Foo`` can only occur strictly positively in the type of any of its recursive arguments. Specifically, in a nested inductive definition, ``Foo`` can appear as an argument to another inductive type constructor, so long as the corresponding parameter occurs strictly positively in the constructors for *that* inductive type. This process can be iterated, so that additional type constructors can be applied to those, and so on.
+
+A nested inductive definition is compiled down to an ordinary inductive definition using a mutual inductive definition to define copies of all the nested types simultaneously. Lean then constructs isomorphisms between the mutually defined nested types and their independently defined counterparts. Once again, the internal details are not meant to be manipulated by users. Rather, the type former and constructors are made available and work as expected, while an appropriate ``sizeOf`` measure is generated for use with well-founded recursion.
+
+```lean
+universe u
+-- BEGIN
+mutual
+inductive Even : Nat → Prop
+| even_zero : Even 0
+| even_succ : ∀ n, Odd n → Even (n + 1)
+inductive Odd : Nat → Prop
+| odd_succ : ∀ n, Even n → Odd (n + 1)
+end
+
+inductive Tree (α : Type u)
+| mk : α → List (Tree α) → Tree α
+
+inductive DoubleTree (α : Type u)
+| mk : α → List (DoubleTree α) × List (DoubleTree α) → DoubleTree α
+-- END
+```
+
+.. _the_equation_compiler:
+
+The Equation Compiler
+=====================
+
+The equation compiler takes an equational description of a function or proof and tries to define an object meeting that specification. It expects input with the following syntax:
+
+```
+def foo (a : α) : Π (b : β), γ
+| [patterns₁] => t₁
+...
+| [patternsₙ] => tₙ
+```
+
+Here ``(a : α)`` is a telescope, ``(b : β)`` is a telescope in the context ``(a : α)``, and ``γ`` is an expression in the context ``(a : α) (b : β)`` denoting a ``Type`` or a ``Prop``.
+
+Each ``patternsᵢ`` is a sequence of patterns of the same length as ``(b : β)``. A pattern is either:
+
+- a variable, denoting an arbitrary value of the relevant type,
+- an underscore, denoting a *wildcard* or *anonymous variable*,
+- an inaccessible term (see below), or
+- a constructor for the inductive type of the corresponding argument, applied to a sequence of patterns.
+
+In the last case, the pattern must be enclosed in parentheses.
+
+Each term ``tᵢ`` is an expression in the context ``(a : α)`` together with the variables introduced on the left-hand side of the token ``=>``. The term ``tᵢ`` can also include recursive calls to ``foo``, as described below. The equation compiler does case splitting on the variables ``(b : β)`` as necessary to match the patterns, and defines ``foo`` so that it has the value ``tᵢ`` in each of the cases. In ideal circumstances (see below), the equations hold definitionally. Whether they hold definitionally or only propositionally, the equation compiler proves the relevant equations and assigns them internal names. They are accessible by the ``rewrite`` and ``simp`` tactics under the name ``foo`` (see [Rewrite](tactics.md#rewrite) and _[TODO: where is simplifier tactic documented?]_. If some of the patterns overlap, the equation compiler interprets the definition so that the first matching pattern applies in each case. Thus, if the last pattern is a variable, it covers all the remaining cases. If the patterns that are presented do not cover all possible cases, the equation compiler raises an error.
+
+When identifiers are marked with the ``[match_pattern]`` attribute, the equation compiler unfolds them in the hopes of exposing a constructor. For example, this makes it possible to write ``n+1`` and ``0`` instead of ``Nat.succ n`` and ``Nat.zero`` in patterns.
+
+For a nonrecursive definition involving case splits, the defining equations will hold definitionally. With inductive types like ``Char``, ``String``, and ``Fin n``, a case split would produce definitions with an inordinate number of cases. To avoid this, the equation compiler uses ``if ... then ... else`` instead of ``casesOn`` when defining the function. In this case, the defining equations hold definitionally as well.
+
+```lean
+open Nat
+
+def sub2 : Nat → Nat
+| zero          => 0
+| succ zero     => 0
+| succ (succ a) => a
+
+def bar : Nat → List Nat → Bool → Nat
+| 0,   _,      false => 0
+| 0,   b :: _, _     => b
+| 0,   [],     true  => 7
+| a+1, [],     false => a
+| a+1, [],     true  => a + 1
+| a+1, b :: _, _     => a + b
+
+def baz : Char → Nat
+| 'A' => 1
+| 'B' => 2
+| _   => 3
+```
+
+The case where patterns are matched against an argument whose type is an inductive family is known as *dependent pattern matching*. This is more complicated, because the type of the function being defined can impose constraints on the patterns that are matched. In this case, the equation compiler will detect inconsistent cases and rule them out.
+
+```lean
+universe u
+
+inductive Vector (α : Type u) : Nat → Type u
+  | nil  : Vector α 0
+  | cons : α → Vector α n → Vector α (n+1)
+
+namespace Vector
+
+def head : Vector α (n+1) → α
+  | cons h t => h
+
+def tail : Vector α (n+1) → Vector α n
+  | cons h t => t
+
+def map (f : α → β → γ) : Vector α n → Vector β n → Vector γ n
+  | nil, nil => nil
+  | cons a va, cons b vb => cons (f a b) (map f va vb)
+
+end Vector
+```
+
+.. _recursive_functions:
+
+Recursive functions
+===================
+
+Lean must ensure that a recursive function terminates, for which there are two strategies: _structural recursion_, in which all recursive calls are made on smaller parts of the input data, and _well-founded recursion_, in which recursive calls are justified by showing that arguments to recursive calls are smaller according to some other measure.
+
+Structural recursion
+--------------------
+
+If the definition of a function contains recursive calls, Lean first tries to interpret the definition as a structural recursion. In order for that to succeed, the recursive arguments must be subterms of the corresponding arguments on the left-hand side.
+
+The function is then defined using a *course of values* recursion, using automatically generated functions ``below`` and ``brec`` in the namespace corresponding to the inductive type of the recursive argument. In this case the defining equations hold definitionally, possibly with additional case splits.
+
+```lean
+namespace Hide
+
+-- BEGIN
+def fib : Nat → Nat
+| 0     => 1
+| 1     => 1
+| (n+2) => fib (n+1) + fib n
+
+def append {α : Type} : List α → List α → List α
+| [],   l => l
+| h::t, l => h :: append t l
+
+example : append [(1 : Nat), 2, 3] [4, 5] = [1, 2, 3, 4, 5] => rfl
+-- END
+
+end Hide
+```
+
+Well-founded recursion
+---------------------
+
+If structural recursion fails, the equation compiler falls back on well-founded recursion. It tries to infer an instance of ``SizeOf`` for the type of each argument, and then tries to find a permutation of the arguments such that each recursive call is decreasing under the lexicographic order with respect to ``sizeOf`` measures.  Lean uses information in the local context, so you can often provide the relevant proof manually using ``have`` in the body of the definition.
+
+In the case of well-founded recursion, the equation used to declare the function holds only propositionally, but not definitionally, and can be accessed using ``unfold``, ``simp`` and ``rewrite`` with the function name (for example ``unfold foo`` or ``simp [foo]``, where ``foo`` is the function defined with well-founded recursion).
+
+```lean
+namespace Hide
+open Nat
+
+-- BEGIN
+def div : Nat → Nat → Nat
+| x, y =>
+  if h : 0 < y ∧ y ≤ x then
+    have : x - y < x :=
+      sub_lt (Nat.lt_of_lt_of_le h.left h.right) h.left
+    div (x - y) y + 1
+  else
+    0
+
+example (x y : Nat) :
+  div x y = if 0 < y ∧ y ≤ x then div (x - y) y + 1 else 0 :=
+by rw [div]; rfl
+-- END
+
+end Hide
+```
+
+If Lean cannot find a permutation of the arguments for which all recursive calls are decreasing, it will print a table that contains, for every recursive call, which arguments Lean could prove to be decreasing. For example, a function with three recursive calls and four parameters might cause the following message to be printed
+
+```
+example.lean:37:0-43:31: error: Could not find a decreasing measure.
+The arguments relate at each recursive call as follows:
+(<, ≤, =: relation proved, ? all proofs failed, _: no proof attempted)
+           x1 x2 x3 x4
+1) 39:6-27  =  =  _  =
+2) 40:6-25  =  ?  _  <
+3) 41:6-25  <  _  _  _
+Please use `termination_by` to specify a decreasing measure.
+```
+
+This table should be read as follows:
+
+ * In the first recursive call, in line 39, arguments 1, 2 and 4 are equal to the function's parameters.
+ * The second recursive call, in line 40, has an equal first argument, a smaller fourth argument, and nothing could be inferred for the second argument.
+ * The third recursive call, in line 41, has a decreasing first argument.
+ * No other proofs were attempted, either because the parameter has a type without a non-trivial ``WellFounded`` instance (parameter 3), or because it is already clear that no decreasing measure can be found.
+
+
+Lean will print the termination argument it found if ``set_option showInferredTerminationBy true`` is set.
+
+If Lean does not find the termination argument, or if you want to be explicit, you can append a `termination_by` clause to the function definition, after the function's body, but before the `where` clause if present. It is of the form
+```
+termination_by e
+```
+where ``e`` is an expression that depends on the parameters of the function and should be decreasing at each recursive call. The type of `e` should be an instance of the class ``WellFoundedRelation``, which determines how to compare two values of that type.
+
+If ``f`` has parameters “after the ``:``” (for example when defining functions via patterns using `|`), then these can be brought into scope using the syntax
+```
+termination_by a₁ … aₙ => e
+```
+
+By default, Lean uses the tactic ``decreasing_tactic`` when proving that an argument is decreasing; see its documentation for how to globally extend it. You can also choose to use a different tactic for a given function definition with the clause
+```
+decreasing_by <tac>
+```
+which should come after ``termination_by`, if present.
+
+
+Note that recursive definitions can in general require nested recursions, that is, recursion on different arguments of ``foo`` in the template above. The equation compiler handles this by abstracting later arguments, and recursively defining higher-order functions to meet the specification.
+
+Mutual recursion
+----------------
+
+The equation compiler also allows mutual recursive definitions, with a syntax similar to that of [Mutual and Nested Inductive Definitions](#mutual-and-nested-inductive-definitions). Mutual definitions are always compiled using well-founded recursion, and so once again the defining equations hold only propositionally.
+
+```lean
+mutual
+def even : Nat → Bool
+| 0   => true
+| a+1 => odd a
+def odd : Nat → Bool
+| 0   => false
+| a+1 => even a
+end
+
+example (a : Nat) : even (a + 1) = odd a :=
+by simp [even]
+
+example (a : Nat) : odd (a + 1) = even a :=
+by simp [odd]
+```
+
+Well-founded recursion is especially useful with [Mutual and Nested Inductive Definitions](#mutual-and-nested-inductive-definitions), since it provides the canonical way of defining functions on these types.
+
+```lean
+mutual
+inductive Even : Nat → Prop
+| even_zero : Even 0
+| even_succ : ∀ n, Odd n → Even (n + 1)
+inductive Odd : Nat → Prop
+| odd_succ : ∀ n, Even n → Odd (n + 1)
+end
+
+open Even Odd
+
+theorem not_odd_zero : ¬ Odd 0 := fun x => nomatch x
+
+mutual
+theorem even_of_odd_succ : ∀ n, Odd (n + 1) → Even n
+| _, odd_succ n h => h
+theorem odd_of_even_succ : ∀ n, Even (n + 1) → Odd n
+| _, even_succ n h => h
+end
+
+inductive Term
+| const : String → Term
+| app   : String → List Term → Term
+
+open Term
+
+mutual
+def num_consts : Term → Nat
+| .const n  => 1
+| .app n ts => num_consts_lst ts
+def num_consts_lst : List Term → Nat
+| []    => 0
+| t::ts => num_consts t + num_consts_lst ts
+end
+```
+
+In a set of mutually recursive function, either all or no functions must have an explicit termination argument (``termination_by``). A change of the default termination tactic (``decreasing_by``) only affects the proofs about the recursive calls of that function, not the other functions in the group.
+
+```
+mutual
+theorem even_of_odd_succ : ∀ n, Odd (n + 1) → Even n
+| _, odd_succ n h => h
+termination_by n h => h
+decreasing_by decreasing_tactic
+
+theorem odd_of_even_succ : ∀ n, Even (n + 1) → Odd n
+| _, even_succ n h => h
+termination_by n h => h
+end
+```
+
+Another way to express mutual recursion is using local function definitions in ``where`` or ``let rec`` clauses: these can be mutually recursive with each other and their containing function:
+
+```
+theorem even_of_odd_succ : ∀ n, Odd (n + 1) → Even n
+  | _, odd_succ n h => h
+termination_by n h => h
+  where
+    theorem odd_of_even_succ : ∀ n, Even (n + 1) → Odd n
+      | _, even_succ n h => h
+    termination_by n h => h
+```
+
+.. _match_expressions:
+
+Match Expressions
+=================
+
+Lean supports a ``match ... with ...`` construct similar to ones found in most functional programming languages. The syntax is as follows:
+
+```
+match t₁, ..., tₙ with
+| p₁₁, ..., p₁ₙ => s₁
+...
+| pₘ₁, ..., pₘₙ => sₘ
+```
+
+Here ``t₁, ..., tₙ`` are any terms in the context in which the expression appears, the expressions ``pᵢⱼ`` are patterns, and the terms ``sᵢ`` are expressions in the local context together with variables introduced by the patterns on the left-hand side. Each ``sᵢ`` should have the expected type of the entire ``match`` expression.
+
+Any ``match`` expression is interpreted using the equation compiler, which generalizes ``t₁, ..., tₙ``, defines an internal function meeting the specification, and then applies it to ``t₁, ..., tₙ``. In contrast to the definitions in [The Equation Compiler](declarations.md#the-equation-compiler), the terms ``tᵢ`` are arbitrary terms rather than just variables, and the expression can occur anywhere within a Lean expression, not just at the top level of a definition. Note that the syntax here is somewhat different: both the terms ``tᵢ`` and the patterns ``pᵢⱼ`` are separated by commas.
+
+```lean
+def foo (n : Nat) (b c : Bool) :=
+5 + match n - 5, b && c with
+    | 0,   true  => 0
+    | m+1, true  => m + 7
+    | 0,   false => 5
+    | m+1, false => m + 3
+```
+
+When a ``match`` has only one line, Lean provides alternative syntax with a destructuring ``let``, as well as a destructuring lambda abstraction. Thus the following definitions all have the same net effect.
+
+```lean
+def bar₁ : Nat × Nat → Nat
+| (m, n) => m + n
+
+def bar₂ (p : Nat × Nat) : Nat :=
+match p with | (m, n) => m + n
+
+def bar₃ : Nat × Nat → Nat :=
+fun ⟨m, n⟩ => m + n
+
+def bar₄ (p : Nat × Nat) : Nat :=
+let ⟨m, n⟩ := p; m + n
+```
+
+Information about the term being matched can be preserved in each branch using the syntax `match h : t with`. For example, a user may want to match a term `ns ++ ms : List Nat`, while tracking the hypothesis `ns ++ ms = []` or `ns ++ ms= h :: t` in the respective match arm:
+
+```lean
+def foo (ns ms : List Nat) (h1 : ns ++ ms ≠ []) (k : Nat -> Char) : Char :=
+  match h2 : ns ++ ms with
+  -- in this arm, we have the hypothesis `h2 : ns ++ ms = []`
+  | [] => absurd h2 h1
+  -- in this arm, we have the hypothesis `h2 : ns ++ ms = h :: t`
+  | h :: t => k h
+
+-- '7'
+#eval foo [7, 8, 9] [] (by decide) Nat.digitChar
+```
+
+.. _structures_and_records:
+
+Structures and Records
+======================
+
+The ``structure`` command in Lean is used to define an inductive data type with a single constructor and to define its projections at the same time. The syntax is as follows:
+
+```
+structure Foo (a : α) extends Bar, Baz : Sort u :=
+constructor :: (field₁ : β₁) ... (fieldₙ : βₙ)
+```
+
+Here ``(a : α)`` is a telescope, that is, the parameters to the inductive definition. The name ``constructor`` followed by the double colon is optional; if it is not present, the name ``mk`` is used by default. The keyword ``extends`` followed by a list of previously defined structures is also optional; if it is present, an instance of each of these structures is included among the fields to ``Foo``, and the types ``βᵢ`` can refer to their fields as well. The output type, ``Sort u``, can be omitted, in which case Lean infers to smallest non-``Prop`` sort possible. Finally, ``(field₁ : β₁) ... (fieldₙ : βₙ)`` is a telescope relative to ``(a : α)`` and the fields in ``bar`` and ``baz``.
+
+The declaration above is syntactic sugar for an inductive type declaration, and so results in the addition of the following constants to the environment:
+
+- the type former : ``Foo : Π (a : α), Sort u``
+- the single constructor :
+
+```
+Foo.constructor : Π (a : α) (toBar : Bar) (toBaz : Baz)
+  (field₁ : β₁) ... (fieldₙ : βₙ), Foo a
+```
+
+- the eliminator ``Foo.rec`` for the inductive type with that constructor
+
+In addition, Lean defines
+
+- the projections : ``fieldᵢ : Π (a : α) (c : Foo) : βᵢ`` for each ``i``
+
+where any other fields mentioned in ``βᵢ`` are replaced by the relevant projections from ``c``.
+
+Given ``c : Foo``, Lean offers the following convenient syntax for the projection ``Foo.fieldᵢ c``:
+
+- *anonymous projections* : ``c.fieldᵢ``
+- *numbered projections* : ``c.i``
+
+These can be used in any situation where Lean can infer that the type of ``c`` is of the form ``Foo a``. The convention for anonymous projections is extended to any function ``f`` defined in the namespace ``Foo``, as described in [Namespaces](namespaces.md).
+
+Similarly, Lean offers the following convenient syntax for constructing elements of ``Foo``. They are equivalent to ``Foo.constructor b₁ b₂ f₁ f₁ ... fₙ``, where ``b₁ : Bar``, ``b₂ : Baz``, and each ``fᵢ : βᵢ`` :
+
+- *anonymous constructor*: ``⟨ b₁, b₂, f₁, ..., fₙ ⟩``
+- *record notation*:
+
+```
+{ toBar := b₁, toBaz := b₂, field₁ := f₁, ...,
+    fieldₙ := fₙ :  Foo a }
+```
+
+The anonymous constructor can be used in any context where Lean can infer that the expression should have a type of the form ``Foo a``. The unicode brackets are entered as ``\<`` and ``\>`` respectively.
+
+When using record notation, you can omit the annotation ``: Foo a`` when Lean can infer that the expression should have a type of the form ``Foo a``. You can replace either ``toBar`` or ``toBaz`` by assignments to *their* fields as well, essentially acting as though the fields of ``Bar`` and ``Baz`` are simply imported into ``Foo``. Finally, record notation also supports
+
+- *record updates*: ``{ t with ... fieldᵢ := fᵢ ...}``
+
+Here ``t`` is a term of type ``Foo a`` for some ``a``. The notation instructs Lean to take values from ``t`` for any field assignment that is omitted from the list.
+
+Lean also allows you to specify a default value for any field in a structure by writing ``(fieldᵢ : βᵢ := t)``. Here ``t`` specifies the value to use when the field ``fieldᵢ`` is left unspecified in an instance of record notation.
+
+```lean
+universe u v
+
+structure Vec (α : Type u) (n : Nat) :=
+(l : List α) (h : l.length = n)
+
+structure Foo (α : Type u) (β : Nat → Type v) : Type (max u v) :=
+(a : α) (n : Nat) (b : β n)
+
+structure Bar :=
+(c : Nat := 8) (d : Nat)
+
+structure Baz extends Foo Nat (Vec Nat), Bar :=
+(v : Vec Nat n)
+
+#check Foo
+#check @Foo.mk
+#check @Foo.rec
+
+#check Foo.a
+#check Foo.n
+#check Foo.b
+
+#check Baz
+#check @Baz.mk
+#check @Baz.rec
+
+#check Baz.toFoo
+#check Baz.toBar
+#check Baz.v
+
+def bzz := Vec.mk [1, 2, 3] rfl
+
+#check Vec.l bzz
+#check Vec.h bzz
+#check bzz.l
+#check bzz.h
+#check bzz.1
+#check bzz.2
+
+example : Vec Nat 3 := Vec.mk [1, 2, 3] rfl
+example : Vec Nat 3 := ⟨[1, 2, 3], rfl⟩
+example : Vec Nat 3 := { l := [1, 2, 3], h := rfl : Vec Nat 3 }
+example : Vec Nat 3 := { l := [1, 2, 3], h := rfl }
+
+example : Foo Nat (Vec Nat) := ⟨1, 3, bzz⟩
+
+example : Baz := ⟨⟨1, 3, bzz⟩, ⟨5, 7⟩, bzz⟩
+example : Baz := { a := 1, n := 3, b := bzz, c := 5, d := 7, v := bzz}
+def fzz : Foo Nat (Vec Nat) := {a := 1, n := 3, b := bzz}
+
+example : Foo Nat (Vec Nat) := { fzz with a := 7 }
+example : Baz := { fzz with c := 5, d := 7, v := bzz }
+
+example : Bar := { c := 8, d := 9 }
+example : Bar := { d := 9 }  -- uses the default value for c
+```
+
+.. _type_classes:
+
+Type Classes
+============
+
+(Classes and instances. Anonymous instances. Local instances.)
+
+
+.. [Dybjer] Dybjer, Peter, *Inductive Families*. Formal Aspects of Computing 6, 1994, pages 440-465.

doc/decltypes.md

@@ -0,0 +1,29 @@
+# Declaring New Types
+
+In Lean's library, every concrete type other than the universes and every type constructor other than the dependent function type is
+an instance of a general family of type constructions known as *inductive types*. It is remarkable that it is possible to develop
+complex programs and formalize mathematics based on nothing more than the type universes, dependent function types,
+and inductive types; everything else follows from those.
+
+Intuitively, an inductive type is built up from a specified list of constructors. In Lean, the basic syntax for specifying such a type is as follows:
+```
+inductive NewType where
+  | constructor_1 : ... → NewType
+  | constructor_2 : ... → NewType
+  ...
+  | constructor_n : ... → NewType
+```
+
+The intuition is that each constructor specifies a way of building new objects of ``NewType``, possibly from previously constructed values.
+The type ``NewType`` consists of nothing more than the objects that are constructed in this way.
+
+We will see below that the arguments to the constructors can include objects of type ``NewType``,
+subject to a certain "positivity" constraint, which guarantees that elements of ``NewType`` are built
+from the bottom up. Roughly speaking, each ``...`` can be any function type constructed from ``NewType``
+and previously defined types, in which ``NewType`` appears, if at all, only as the "target" of the function type.
+
+We will provide a number of examples of inductive types. We will also consider slight generalizations of the scheme above,
+to mutually defined inductive types, and so-called *inductive families*.
+
+Every inductive type comes with constructors, which show how to construct an element of the type, and elimination rules,
+which show how to "use" an element of the type in another construction.

doc/definitions.md

@@ -0,0 +1 @@
+# Definitions

doc/dep.md

@@ -0,0 +1,66 @@
+## What makes dependent type theory dependent?
+
+The short explanation is that what makes dependent type theory dependent is that types can depend on parameters.
+You have already seen a nice example of this: the type ``List α`` depends on the argument ``α``, and
+this dependence is what distinguishes ``List Nat`` and ``List Bool``.
+For another example, consider the type ``Vector α n``, the type of vectors of elements of ``α`` of length ``n``.
+This type depends on *two* parameters: the type ``α : Type`` of the elements in the vector and the length ``n : Nat``.
+
+Suppose we wish to write a function ``cons`` which inserts a new element at the head of a list.
+What type should ``cons`` have? Such a function is *polymorphic*: we expect the ``cons`` function for ``Nat``, ``Bool``,
+or an arbitrary type ``α`` to behave the same way.
+So it makes sense to take the type to be the first argument to ``cons``, so that for any type, ``α``, ``cons α``
+is the insertion function for lists of type ``α``. In other words, for every ``α``, ``cons α`` is the function that takes an element ``a : α``
+and a list ``as : List α``, and returns a new list, so we have ``cons α a as : list α``.
+
+It is clear that ``cons α`` should have type ``α → List α → List α``. But what type should ``cons`` have?
+A first guess might be ``Type → α → list α → list α``, but, on reflection, this does not make sense:
+the ``α`` in this expression does not refer to anything, whereas it should refer to the argument of type ``Type``.
+In other words, *assuming* ``α : Type`` is the first argument to the function, the type of the next two elements are ``α`` and ``List α``.
+These types vary depending on the first argument, ``α``.
+
+This is an instance of a *dependent function type*, or *dependent arrow type*. Given ``α : Type`` and ``β : α → Type``,
+think of ``β`` as a family of types over ``α``, that is, a type ``β a`` for each ``a : α``.
+In that case, the type ``(a : α) → β a`` denotes the type of functions ``f`` with the property that,
+for each ``a : α``, ``f a`` is an element of ``β a``. In other words, the type of the value returned by ``f`` depends on its input.
+
+Notice that ``(a : α) → β`` makes sense for any expression ``β : Type``. When the value of ``β`` depends on ``a``
+(as does, for example, the expression ``β a`` in the previous paragraph), ``(a : α) → β`` denotes a dependent function type.
+When ``β`` doesn't depend on ``a``, ``(a : α) → β`` is no different from the type ``α → β``.
+Indeed, in dependent type theory (and in Lean), ``α → β`` is just notation for ``(a : α) → β`` when ``β`` does not depend on ``a``.
+
+Returning to the example of lists, we can use the command `#check` to inspect the type of the following `List` functions
+We will explain the ``@`` symbol and the difference between the round and curly braces momentarily.
+
+```lean
+#check @List.cons    -- {α : Type u_1} → α → List α → List α
+#check @List.nil     -- {α : Type u_1} → List α
+#check @List.length  -- {α : Type u_1} → List α → Nat
+#check @List.append  -- {α : Type u_1} → List α → List α → List α
+```
+
+Just as dependent function types ``(a : α) → β a`` generalize the notion of a function type ``α → β`` by allowing ``β`` to depend on ``α``,
+dependent Cartesian product types ``(a : α) × β a`` generalize the Cartesian product ``α × β`` in the same way. Dependent products are also
+called *sigma* types, and you can also write them as `Σ a : α, β a`. You can use `⟨a, b⟩` or `Sigma.mk a b` to create a dependent pair.
+
+```lean
+universe u v
+
+def f (α : Type u) (β : α → Type v) (a : α) (b : β a) : (a : α) × β a :=
+  ⟨a, b⟩
+
+def g (α : Type u) (β : α → Type v) (a : α) (b : β a) : Σ a : α, β a :=
+  Sigma.mk a b
+
+#reduce f
+#reduce g
+
+#reduce f Type (fun α => α) Nat 10
+#reduce g Type (fun α => α) Nat 10
+
+#reduce (f Type (fun α => α) Nat 10).1 -- Nat
+#reduce (g Type (fun α => α) Nat 10).1 -- Nat
+#reduce (f Type (fun α => α) Nat 10).2 -- 10
+#reduce (g Type (fun α => α) Nat 10).2 -- 10
+```
+The function `f` and `g` above denote the same function.

doc/deptypes.md

@@ -0,0 +1,3 @@
+# Dependent Types
+
+In this section, we introduce simple type theory, types as objects, definitions, and explain what makes dependent type theory *dependent*.

doc/dev/bootstrap.md

@@ -73,7 +73,7 @@ update the archived C source code of the stage 0 compiler in `stage0/src`.
 The github repository will automatically update stage0 on `master` once
 `src/stdlib_flags.h` and `stage0/src/stdlib_flags.h` are out of sync.
 
-If you have write access to the lean4 repository, you can also manually
+If you have write access to the lean4 repository, you can also also manually
 trigger that process, for example to be able to use new features in the compiler itself.
 You can do that on <https://github.com/leanprover/lean4/actions/workflows/update-stage0.yml>
 or using Github CLI with
@@ -103,21 +103,10 @@ your PR using rebase merge, bypassing the merge queue.
 As written above, changes in meta code in the current stage usually will only
 affect later stages. This is an issue in two specific cases.
 
-* For the special case of *quotations*, it is desirable to have changes in builtin parsers affect them immediately: when the changes in the parser become active in the next stage, builtin macros implemented via quotations should generate syntax trees compatible with the new parser, and quotation patterns in builtin macros and elaborators should be able to match syntax created by the new parser and macros.
-  Since quotations capture the syntax tree structure during execution of the current stage and turn it into code for the next stage, we need to run the current stage's builtin parsers in quotations via the interpreter for this to work.
-  Caveats:
-  * We activate this behavior by default when building stage 1 by setting `-Dinternal.parseQuotWithCurrentStage=true`.
-    We force-disable it inside `macro/macro_rules/elab/elab_rules` via `suppressInsideQuot` as they are guaranteed not to run in the next stage and may need to be run in the current one, so the stage 0 parser is the correct one to use for them.
-    It may be necessary to extend this disabling to functions that contain quotations and are (exclusively) used by one of the mentioned commands. A function using quotations should never be used by both builtin and non-builtin macros/elaborators. Example: https://github.com/leanprover/lean4/blob/f70b7e5722da6101572869d87832494e2f8534b7/src/Lean/Elab/Tactic/Config.lean#L118-L122
-  * The parser needs to be reachable via an `import` statement, otherwise the version of the previous stage will silently be used.
-  * Only the parser code (`Parser.fn`) is affected; all metadata such as leading tokens is taken from the previous stage.
-
-  For an example, see https://github.com/leanprover/lean4/commit/f9dcbbddc48ccab22c7674ba20c5f409823b4cc1#diff-371387aed38bb02bf7761084fd9460e4168ae16d1ffe5de041b47d3ad2d22422R13
-
 * For *non-builtin* meta code such as `notation`s or `macro`s in
   `Notation.lean`, we expect changes to affect the current file and all later
   files of the same stage immediately, just like outside the stdlib. To ensure
-  this, we build stage 1 using `-Dinterpreter.prefer_native=false` -
+  this, we need to build the stage using `-Dinterpreter.prefer_native=false` -
   otherwise, when executing a macro, the interpreter would notice that there is
   already a native symbol available for this function and run it instead of the
   new IR, but the symbol is from the previous stage!
@@ -135,11 +124,26 @@ affect later stages. This is an issue in two specific cases.
   further stages (e.g. after an `update-stage0`) will then need to be compiled
   with the flag set to `false` again since they will expect the new signature.
 
-  When enabling `prefer_native`, we usually want to *disable* `parseQuotWithCurrentStage` as it would otherwise make quotations use the interpreter after all.
-  However, there is a specific case where we want to set both options to `true`: when we make changes to a non-builtin parser like `simp` that has a builtin elaborator, we cannot have the new parser be active outside of quotations in stage 1 as the builtin elaborator from stage 0 would not understand them; on the other hand, we need quotations in e.g. the builtin `simp` elaborator to produce the new syntax in the next stage.
-  As this issue usually affects only tactics, enabling `debug.byAsSorry` instead of `prefer_native` can be a simpler solution.
+  For an example, see https://github.com/leanprover/lean4/commit/da4c46370d85add64ef7ca5e7cc4638b62823fbb.
 
-  For a `prefer_native` example, see https://github.com/leanprover/lean4/commit/da4c46370d85add64ef7ca5e7cc4638b62823fbb.
+* For the special case of *quotations*, it is desirable to have changes in
+  built-in parsers affect them immediately: when the changes in the parser
+  become active in the next stage, macros implemented via quotations should
+  generate syntax trees compatible with the new parser, and quotation patterns
+  in macro and elaborators should be able to match syntax created by the new
+  parser and macros. Since quotations capture the syntax tree structure during
+  execution of the current stage and turn it into code for the next stage, we
+  need to run the current stage's built-in parsers in quotation via the
+  interpreter for this to work. Caveats:
+  * Since interpreting full parsers is not nearly as cheap and we rarely change
+    built-in syntax, this needs to be opted in using `-Dinternal.parseQuotWithCurrentStage=true`.
+  * The parser needs to be reachable via an `import` statement, otherwise the
+    version of the previous stage will silently be used.
+  * Only the parser code (`Parser.fn`) is affected; all metadata such as leading
+    tokens is taken from the previous stage.
+
+  For an example, see https://github.com/leanprover/lean4/commit/f9dcbbddc48ccab22c7674ba20c5f409823b4cc1#diff-371387aed38bb02bf7761084fd9460e4168ae16d1ffe5de041b47d3ad2d22422
+  (from before the flag defaulted to `false`).
 
 To modify either of these flags both for building and editing the stdlib, adjust
 the code in `stage0/src/stdlib_flags.h`. The flags will automatically be reset

doc/dev/commit_convention.md

@@ -33,9 +33,6 @@ Format of the commit message
  - chore (maintain, ex: travis-ci)
  - perf (performance improvement, optimization, ...)
 
-Every `feat` or `fix` commit must have a `changelog-*` label, and a commit message
-beginning with "This PR " that will be included in the changelog.
-
 ``<subject>`` has the following constraints:
 
  - use imperative, present tense: "change" not "changed" nor "changes"
@@ -47,7 +44,6 @@ beginning with "This PR " that will be included in the changelog.
  - just as in ``<subject>``, use imperative, present tense
  - includes motivation for the change and contrasts with previous
    behavior
- - If a `changelog-*` label is present, the body must begin with "This PR ".
 
 ``<footer>`` is optional and may contain two items:
 
@@ -64,21 +60,17 @@ Examples
 
 fix: add declarations for operator<<(std::ostream&, expr const&) and operator<<(std::ostream&, context const&) in the kernel
 
-This PR adds declarations `operator<<` for raw printing.
 The actual implementation of these two operators is outside of the
-kernel. They are implemented in the file 'library/printer.cpp'.
-
-We declare them in the kernel to prevent the following problem.
-Suppose there is a file 'foo.cpp' that does not include 'library/printer.h',
+kernel. They are implemented in the file 'library/printer.cpp'. We
+declare them in the kernel to prevent the following problem. Suppose
+there is a file 'foo.cpp' that does not include 'library/printer.h',
 but contains
-```cpp
-expr a;
-...
-std::cout << a << "\n";
-...
-```
+
+    expr a;
+    ...
+    std::cout << a << "\n";
+    ...
 
 The compiler does not generate an error message. It silently uses the
 operator bool() to coerce the expression into a Boolean. This produces
 counter-intuitive behavior, and may confuse developers.
-

doc/dev/ffi.md

@@ -49,9 +49,8 @@ In the case of `@[extern]` all *irrelevant* types are removed first; see next se
   is represented by the representation of that parameter's type.
 
   For example, `{ x : α // p }`, the `Subtype` structure of a value of type `α` and an irrelevant proof, is represented by the representation of `α`.
-  Similarly, the signed integer types `Int8`, ..., `Int64`, `ISize` are also represented by the unsigned C types `uint8_t`, ..., `uint64_t`, `size_t`, respectively, because they have a trivial structure.
-* `Nat` and `Int` are represented by `lean_object *`.
-  Their runtime values is either a pointer to an opaque bignum object or, if the lowest bit of the "pointer" is 1 (`lean_is_scalar`), an encoded unboxed natural number or integer (`lean_box`/`lean_unbox`).
+* `Nat` is represented by `lean_object *`.
+  Its runtime value is either a pointer to an opaque bignum object or, if the lowest bit of the "pointer" is 1 (`lean_is_scalar`), an encoded unboxed natural number (`lean_box`/`lean_unbox`).
 * A universe `Sort u`, type constructor `... → Sort u`, or proposition `p : Prop` is *irrelevant* and is either statically erased (see above) or represented as a `lean_object *` with the runtime value `lean_box(0)`
 * Any other type is represented by `lean_object *`.
   Its runtime value is a pointer to an object of a subtype of `lean_object` (see the "Inductive types" section below) or the unboxed value `lean_box(cidx)` for the `cidx`th constructor of an inductive type if this constructor does not have any relevant parameters.
@@ -68,7 +67,7 @@ The memory order of the fields is derived from the types and order of the fields
 * Fields of type `USize`
 * Other scalar fields, in decreasing order by size
 
-Within each group the fields are ordered in declaration order. Trivial wrapper types count as their underlying wrapped type for this purpose.
+Within each group the fields are ordered in declaration order. **Warning**: Trivial wrapper types still count toward a field being treated as non-scalar for this purpose.
 
 * To access fields of the first kind, use `lean_ctor_get(val, i)` to get the `i`th non-scalar field.
 * To access `USize` fields, use `lean_ctor_get_usize(val, n+i)` to get the `i`th usize field and `n` is the total number of fields of the first kind.
@@ -80,32 +79,32 @@ structure S where
   ptr_1 : Array Nat
   usize_1 : USize
   sc64_1 : UInt64
-  sc64_2 : { x : UInt64 // x > 0 } -- wrappers of scalars count as scalars
-  sc64_3 : Float -- `Float` is 64 bit
+  ptr_2 : { x : UInt64 // x > 0 } -- wrappers don't count as scalars
+  sc64_2 : Float -- `Float` is 64 bit
   sc8_1 : Bool
   sc16_1 : UInt16
   sc8_2 : UInt8
-  sc64_4 : UInt64
+  sc64_3 : UInt64
   usize_2 : USize
-  sc32_1 : Char -- trivial wrapper around `UInt32`
-  sc32_2 : UInt32
+  ptr_3 : Char -- trivial wrapper around `UInt32`
+  sc32_1 : UInt32
   sc16_2 : UInt16
 ```
 would get re-sorted into the following memory order:
 
 * `S.ptr_1` - `lean_ctor_get(val, 0)`
-* `S.usize_1` - `lean_ctor_get_usize(val, 1)`
-* `S.usize_2` - `lean_ctor_get_usize(val, 2)`
-* `S.sc64_1` - `lean_ctor_get_uint64(val, sizeof(void*)*3)`
-* `S.sc64_2` - `lean_ctor_get_uint64(val, sizeof(void*)*3 + 8)`
-* `S.sc64_3` - `lean_ctor_get_float(val, sizeof(void*)*3 + 16)`
-* `S.sc64_4` - `lean_ctor_get_uint64(val, sizeof(void*)*3 + 24)`
-* `S.sc32_1` - `lean_ctor_get_uint32(val, sizeof(void*)*3 + 32)`
-* `S.sc32_2` - `lean_ctor_get_uint32(val, sizeof(void*)*3 + 36)`
-* `S.sc16_1` - `lean_ctor_get_uint16(val, sizeof(void*)*3 + 40)`
-* `S.sc16_2` - `lean_ctor_get_uint16(val, sizeof(void*)*3 + 42)`
-* `S.sc8_1` - `lean_ctor_get_uint8(val, sizeof(void*)*3 + 44)`
-* `S.sc8_2` - `lean_ctor_get_uint8(val, sizeof(void*)*3 + 45)`
+* `S.ptr_2` - `lean_ctor_get(val, 1)`
+* `S.ptr_3` - `lean_ctor_get(val, 2)`
+* `S.usize_1` - `lean_ctor_get_usize(val, 3)`
+* `S.usize_2` - `lean_ctor_get_usize(val, 4)`
+* `S.sc64_1` - `lean_ctor_get_uint64(val, sizeof(void*)*5)`
+* `S.sc64_2` - `lean_ctor_get_float(val, sizeof(void*)*5 + 8)`
+* `S.sc64_3` - `lean_ctor_get_uint64(val, sizeof(void*)*5 + 16)`
+* `S.sc32_1` - `lean_ctor_get_uint32(val, sizeof(void*)*5 + 24)`
+* `S.sc16_1` - `lean_ctor_get_uint16(val, sizeof(void*)*5 + 28)`
+* `S.sc16_2` - `lean_ctor_get_uint16(val, sizeof(void*)*5 + 30)`
+* `S.sc8_1` - `lean_ctor_get_uint8(val, sizeof(void*)*5 + 32)`
+* `S.sc8_2` - `lean_ctor_get_uint8(val, sizeof(void*)*5 + 33)`
 
 ### Borrowing
 
@@ -131,23 +130,16 @@ Thus `[init]` functions are run iff their module is imported, regardless of whet
 
 The initializer for module `A.B` is called `initialize_A_B` and will automatically initialize any imported modules.
 Module initializers are idempotent (when run with the same `builtin` flag), but not thread-safe.
-
-**Important for process-related functionality**: If your application needs to use process-related functions from libuv, such as `Std.Internal.IO.Process.getProcessTitle` and `Std.Internal.IO.Process.setProcessTitle`, you must call `lean_setup_args(argc, argv)` (which returns a potentially modified `argv` that must be used in place of the original) **before** calling `lean_initialize()` or `lean_initialize_runtime_module()`. This sets up process handling capabilities correctly, which is essential for certain system-level operations that Lean's runtime may depend on.
-
 Together with initialization of the Lean runtime, you should execute code like the following exactly once before accessing any Lean declarations:
-
 ```c
 void lean_initialize_runtime_module();
 void lean_initialize();
-char ** lean_setup_args(int argc, char ** argv);
-
 lean_object * initialize_A_B(uint8_t builtin, lean_object *);
 lean_object * initialize_C(uint8_t builtin, lean_object *);
 ...
 
-argv = lean_setup_args(argc, argv); // if using process-related functionality
 lean_initialize_runtime_module();
-//lean_initialize();  // necessary (and replaces `lean_initialize_runtime_module`) if you (indirectly) access the `Lean` package
+//lean_initialize();  // necessary if you (indirectly) access the `Lean` package
 
 lean_object * res;
 // use same default as for Lean executables

doc/dev/index.md

@@ -80,18 +80,3 @@ Unlike most Lean projects, all submodules of the `Lean` module begin with the
 `prelude` keyword. This disables the automated import of `Init`, meaning that
 developers need to figure out their own subset of `Init` to import. This is done
 such that changing files in `Init` doesn't force a full rebuild of `Lean`.
-
-### Testing against Mathlib/Batteries
-You can test a Lean PR against Mathlib and Batteries by rebasing your PR
-on to `nightly-with-mathlib` branch. (It is fine to force push after rebasing.)
-CI will generate a branch of Mathlib and Batteries called `lean-pr-testing-NNNN`
-on the `leanprover-community/mathlib4-nightly-testing` fork of Mathlib.
-This branch uses the toolchain for your PR, and will report back to the Lean PR with results from Mathlib CI.
-See https://leanprover-community.github.io/contribute/tags_and_branches.html for more details.
-
-### Testing against the Lean Language Reference
-You can test a Lean PR against the reference manual by rebasing your PR
-on to `nightly-with-manual` branch. (It is fine to force push after rebasing.)
-CI will generate a branch of the reference manual called `lean-pr-testing-NNNN`
-in `leanprover/reference-manual`. This branch uses the toolchain for your PR,
-and will report back to the Lean PR with results from Mathlib CI.

doc/dev/mdbook.md

@@ -0,0 +1,109 @@
+# Documentation
+
+The Lean `doc` folder contains the [Lean Manual](https://lean-lang.org/lean4/doc/) and is
+authored in a combination of markdown (`*.md`) files and literate Lean files.  The .lean files are
+preprocessed using a tool called [LeanInk](https://github.com/leanprover/leanink) and
+[Alectryon](https://github.com/Kha/alectryon) which produces a generated markdown file.  We then run
+`mdbook` on the result to generate the html pages.
+
+
+## Settings
+
+We are using the following settings while editing the markdown docs.
+
+```json
+{
+    "files.insertFinalNewline": true,
+    "files.trimTrailingWhitespace": true,
+    "[markdown]": {
+        "rewrap.wrappingColumn": 70
+    }
+}
+```
+
+## Build
+
+### Using Nix
+
+Building the manual using Nix (which is what the CI does) is as easy as
+```bash
+$ nix build --update-input lean ./doc
+```
+You can also open a shell with `mdbook` for running the commands mentioned below with
+`nix develop ./doc#book`. Otherwise, read on.
+
+### Manually
+
+To build and test the book you have to preprocess the .lean files with Alectryon then use our own
+fork of the Rust tool named [mdbook](https://github.com/leanprover/mdbook). We have our own fork of
+mdBook with the following additional features:
+
+* Add support for hiding lines in other languages
+  [#1339](https://github.com/rust-lang/mdBook/pull/1339)
+* Make `mdbook test` call the `lean` compiler to test the snippets.
+* Ability to test a single chapter at a time which is handy when you
+are working on that chapter.  See the `--chapter` option.
+
+So you need to setup these tools before you can run `mdBook`.
+
+1. install [Rust](https://www.rust-lang.org/tools/install)
+which provides you with the `cargo` tool for building rust packages.
+Then run the following:
+    ```bash
+    cargo install --git https://github.com/leanprover/mdBook mdbook
+    ```
+
+1. Clone https://github.com/leanprover/LeanInk.git and run `lake build` then make the resulting
+   binary available to Alectryon using e.g.
+    ```bash
+    # make `leanInk` available in the current shell
+    export PATH=$PWD/build/bin:$PATH
+    ```
+
+1. Create a Python 3.10 environment.
+
+1. Install Alectryon:
+    ```
+    python3 -m pip install git+https://github.com/Kha/alectryon.git@typeid
+    ```
+
+1. Now you are ready to process the `*.lean` files using Alectryon as follows:
+
+    ```
+    cd lean4/doc
+    alectryon --frontend lean4+markup examples/palindromes.lean --backend webpage -o palindromes.lean.md
+    ```
+
+    Repeat this for the other .lean files you care about or write a script to process them all.
+
+1. Now you can build the book using:
+    ```
+    cd lean4/doc
+    mdbook build
+    ```
+
+This will put the HTML in a `out` folder so you can load `out/index.html` in your web browser and
+it should look like https://lean-lang.org/lean4/doc/.
+
+1. It is also handy to use e.g. [`mdbook watch`](https://rust-lang.github.io/mdBook/cli/watch.html)
+   in the `doc/` folder so that it keeps the html up to date while you are editing.
+
+    ```bash
+    mdbook watch --open  # opens the output in `out/` in your default browser
+    ```
+
+## Testing Lean Snippets
+
+You can run the following in the `doc/` folder to test all the lean code snippets.
+
+    ```bash
+    mdbook test
+    ```
+
+and you can use the `--chapter` option to test a specific chapter that you are working on:
+
+    ```bash
+    mdbook test --chapter Array
+    ```
+
+Use chapter name `?` to get a list of all the chapter names.

doc/dev/release_checklist.md

@@ -5,105 +5,122 @@ See below for the checklist for release candidates.
 
 We'll use `v4.6.0` as the intended release version as a running example.
 
-- Run `script/release_checklist.py v4.6.0` to check the status of the release.
-  This script is idempotent, and should be safe to run at any stage of the release process.
-  Note that as of v4.19.0, this script takes some autonomous actions, which can be prevented via `--dry-run`.
+- One week before the planned release, ensure that
+  (1) someone has written the release notes and
+  (2) someone has written the first draft of the release blog post.
+  If there is any material in `./releases_drafts/` on the `releases/v4.6.0` branch, then the release notes are not done.
+  (See the section "Writing the release notes".)
 - `git checkout releases/v4.6.0`
   (This branch should already exist, from the release candidates.)
 - `git pull`
 - In `src/CMakeLists.txt`, verify you see
   - `set(LEAN_VERSION_MINOR 6)` (for whichever `6` is appropriate)
   - `set(LEAN_VERSION_IS_RELEASE 1)`
-  - (all of these should already be in place from the release candidates)
+  - (both of these should already be in place from the release candidates)
 - `git tag v4.6.0`
 - `git push $REMOTE v4.6.0`, where `$REMOTE` is the upstream Lean repository (e.g., `origin`, `upstream`)
 - Now wait, while CI runs.
   - You can monitor this at `https://github.com/leanprover/lean4/actions/workflows/ci.yml`,
     looking for the `v4.6.0` tag.
-  - This step can take up to two hours.
+  - This step can take up to an hour.
   - If you are intending to cut the next release candidate on the same day,
     you may want to start on the release candidate checklist now.
-- Next we need to prepare the release notes.
-  - If the stable release is identical to the last release candidate (this should usually be the case),
-    you can reuse the release notes that are already in the Lean Language Reference.
-  - If you want to regenerate the release notes,
-    run `script/release_notes.py --since v4.5.0` on the `releases/v4.6.0` branch,
-    and see the section "Writing the release notes" below for more information.
-  - Release notes live in https://github.com/leanprover/reference-manual, in e.g. `Manual/Releases/v4.6.0.lean`.
-    It's best if you update these at the same time as a you update the `lean-toolchain` for the `reference-manual` repository, see below.
 - Go to https://github.com/leanprover/lean4/releases and verify that the `v4.6.0` release appears.
-  - Verify on Github that "Set as the latest release" is checked.
+  - Edit the release notes on Github to select the "Set as the latest release".
+  - Follow the instructions in creating a release candidate for the "GitHub release notes" step,
+    now that we have a written `RELEASES.md` section.
+    Do a quick sanity check.
 - Next, we will move a curated list of downstream repos to the latest stable release.
-  - In order to have the access rights to push to these repositories and merge PRs,
-    you will need to be a member of the `lean-release-managers` team at both `leanprover-community` and `leanprover`.
-    Contact Kim Morrison (@kim-em) to arrange access.
-  - For each of the repositories listed in `script/release_repos.yml`,
-    - Run `script/release_steps.py v4.6.0 <repo>` (e.g. replacing `<repo>` with `batteries`), which will walk you through the following steps:
-      - Create a new branch off `master`/`main` (as specified in the `branch` field), called `bump_to_v4.6.0`.
-      - Update the contents of `lean-toolchain` to `leanprover/lean4:v4.6.0`.
-      - In the `lakefile.toml` or `lakefile.lean`, if there are dependencies on specific version tags of dependencies, update them to the new tag.
-        If they depend on `main` or `master`, don't change this; you've just updated the dependency, so `lake update` will take care of modifying the manifest.
+  - For each of the repositories listed below:
+    - Make a PR to `master`/`main` changing the toolchain to `v4.6.0`
+      - Update the toolchain file
+      - In the Lakefile, if there are dependencies on specific version tags of dependencies that you've already pushed as part of this process, update them to the new tag.
+        If they depend on `main` or `master`, don't change this; you've just updated the dependency, so it will work and be saved in the manifest
       - Run `lake update`
-      - Commit the changes as `chore: bump toolchain to v4.6.0` and push.
-      - Create a PR with title "chore: bump toolchain to v4.6.0".
+      - The PR title should be "chore: bump toolchain to v4.6.0".
     - Merge the PR once CI completes.
-    - Re-running `script/release_checklist.py` will then create the tag `v4.6.0` from `master`/`main` and push it (unless `toolchain-tag: false` in the `release_repos.yml` file)
-    - `script/release_checklist.py` will then merge the tag `v4.6.0` into the `stable` branch and push it (unless `stable-branch: false` in the `release_repos.yml` file).
-  - Special notes on repositories with exceptional requirements:
-    - `doc-gen4` has additional dependencies which we do not update at each toolchain release, although occasionally these break and need to be updated manually.
-    - `verso`:
-      - The `subverso` dependency is unusual in that it needs to be compatible with _every_ Lean release simultaneously.
-        Usually you don't need to do anything.
-        If you think something is wrong here please contact David Thrane Christiansen (@david-christiansen)
-      - Warnings during `lake update` and `lake build` are expected.
-    - `reference-manual`: the release notes generated by `script/release_notes.py` as described above must be included in
-      `Manual/Releases/v4.6.0.lean`, and `import` and `include` statements adding in `Manual/Releases.lean`. 
-    - `ProofWidgets4` uses a non-standard sequential version tagging scheme, e.g. `v0.0.29`, which does not refer to the toolchain being used.
-      You will need to identify the next available version number from https://github.com/leanprover-community/ProofWidgets4/releases,
-      and push a new tag after merging the PR to `main`.
-    - `mathlib4`:
-      - The `lakefile.toml` should always refer to dependencies via their `main` or `master` branch,
-        not a toolchain tag
-        (with the exception of `ProofWidgets4`, which *must* use a sequential version tag).
-      - Push the PR branch to the main Mathlib repository rather than a fork, or CI may not work reliably
-    - `repl`:
-      There are two copies of `lean-toolchain`/`lakefile.lean`:
-      in the root, and in `test/Mathlib/`. Edit both, and run `lake update` in both directories.
-- An awkward situation that sometimes occurs (e.g. with Verso) is that the `master`/`main` branch has already been moved
-  to a nightly toolchain that comes *after* the stable toolchain we are
-  targeting. In this case it is necessary to create a branch `releases/v4.6.0` from the last commit which was on
-  an earlier toolchain, move that branch to the stable toolchain, and create the toolchain tag from that branch.
-- Run `script/release_checklist.py v4.6.0` one last time to check that everything is in order.
+    - Create the tag `v4.6.0` from `master`/`main` and push it.
+    - Merge the tag `v4.6.0` into the `stable` branch and push it.
+  - We do this for the repositories:
+    - [lean4checker](https://github.com/leanprover/lean4checker)
+      - No dependencies
+      - Toolchain bump PR
+      - Create and push the tag
+      - Merge the tag into `stable`
+    - [Batteries](https://github.com/leanprover-community/batteries)
+      - No dependencies
+      - Toolchain bump PR
+      - Create and push the tag
+      - Merge the tag into `stable`
+    - [ProofWidgets4](https://github.com/leanprover-community/ProofWidgets4)
+      - Dependencies: `Batteries`
+      - Note on versions and branches:
+        - `ProofWidgets` uses a sequential version tagging scheme, e.g. `v0.0.29`,
+          which does not refer to the toolchain being used.
+        - Make a new release in this sequence after merging the toolchain bump PR.
+        - `ProofWidgets` does not maintain a `stable` branch.
+      - Toolchain bump PR
+      - Create and push the tag, following the version convention of the repository
+    - [Aesop](https://github.com/leanprover-community/aesop)
+      - Dependencies: `Batteries`
+      - Toolchain bump PR including updated Lake manifest
+      - Create and push the tag
+      - Merge the tag into `stable`
+    - [doc-gen4](https://github.com/leanprover/doc-gen4)
+      - Dependencies: exist, but they're not part of the release workflow
+      - Toolchain bump PR including updated Lake manifest
+      - Create and push the tag
+      - There is no `stable` branch; skip this step
+    - [import-graph](https://github.com/leanprover-community/import-graph)
+      - Toolchain bump PR including updated Lake manifest
+      - Create and push the tag
+      - There is no `stable` branch; skip this step
+    - [Mathlib](https://github.com/leanprover-community/mathlib4)
+      - Dependencies: `Aesop`, `ProofWidgets4`, `lean4checker`, `Batteries`, `doc-gen4`, `import-graph`
+      - Toolchain bump PR notes:
+        - In addition to updating the `lean-toolchain` and `lakefile.lean`,
+          in `.github/workflows/lean4checker.yml` update the line
+          `git checkout v4.6.0` to the appropriate tag. 
+        - Push the PR branch to the main Mathlib repository rather than a fork, or CI may not work reliably
+        - Create and push the tag
+        - Create a new branch from the tag, push it, and open a pull request against `stable`.
+          Coordinate with a Mathlib maintainer to get this merged.
+    - [REPL](https://github.com/leanprover-community/repl)
+      - Dependencies: `Mathlib` (for test code)
+      - Note that there are two copies of `lean-toolchain`/`lakefile.lean`:
+        in the root, and in `test/Mathlib/`. Edit both, and run `lake update` in both directories.
+      - Toolchain bump PR including updated Lake manifest
+      - Create and push the tag
+      - Merge the tag into `stable`
+- The `v4.6.0` section of `RELEASES.md` is out of sync between
+  `releases/v4.6.0` and `master`. This should be reconciled:
+  - Replace the `v4.6.0` section on `master` with the `v4.6.0` section on `releases/v4.6.0`
+    and commit this to `master`.
+- Merge the release announcement PR for the Lean website - it will be deployed automatically
 - Finally, make an announcement!
   This should go in https://leanprover.zulipchat.com/#narrow/stream/113486-announce, with topic `v4.6.0`.
   Please see previous announcements for suggested language.
   You will want a few bullet points for main topics from the release notes.
-  If there is a blog post, link to that from the zulip announcement.
+  Link to the blog post from the Zulip announcement.
 - Make sure that whoever is handling social media knows the release is out.
 
-## Time estimates:
-- Initial checks and push the tag: 10 minutes.
-- Waiting for the release: 120 minutes.
-- Preparing release notes: 10 minutes.
-- Bumping toolchains in downstream repositories, up to creating the Mathlib PR: 60 minutes.
+## Optimistic(?) time estimates:
+- Initial checks and push the tag: 30 minutes.
+- Waiting for the release: 60 minutes.
+- Fixing release notes: 10 minutes.
+- Bumping toolchains in downstream repositories, up to creating the Mathlib PR: 30 minutes.
 - Waiting for Mathlib CI and bors: 120 minutes.
-- Finalizing Mathlib tags and stable branch, and updating REPL: 20 minutes.
-- Posting announcement and/or blog post: 30 minutes.
+- Finalizing Mathlib tags and stable branch, and updating REPL: 15 minutes.
+- Posting announcement and/or blog post: 20 minutes.
 
 # Creating a release candidate.
 
 This checklist walks you through creating the first release candidate for a version of Lean.
 
-For subsequent release candidates, the process is essentially the same, but we start out with the `releases/v4.7.0` branch already created.
-
 We'll use `v4.7.0-rc1` as the intended release version in this example.
 
 - Decide which nightly release you want to turn into a release candidate.
   We will use `nightly-2024-02-29` in this example.
-- It is essential to choose the nightly that will become the release candidate as early as possible, to avoid confusion.
-- Throughout this process you can use `script/release_checklist.py v4.7.0-rc1` to track progress.
-  This script will also try to do some steps autonomously. It is idempotent and safe to run at any point.
-  You can prevent it taking any actions using `--dry-run`.
 - It is essential that Batteries and Mathlib already have reviewed branches compatible with this nightly.
   - Check that both Batteries and Mathlib's `bump/v4.7.0` branch contain `nightly-2024-02-29`
     in their `lean-toolchain`.
@@ -114,68 +131,79 @@ We'll use `v4.7.0-rc1` as the intended release version in this example.
     git fetch nightly tag nightly-2024-02-29
     git checkout nightly-2024-02-29
     git checkout -b releases/v4.7.0
-    git push --set-upstream origin releases/v4.7.0
     ```
+- In `RELEASES.md` replace `Development in progress` in the `v4.7.0` section with `Release notes to be written.`
+- We will rely on automatically generated release notes for release candidates,
+  and the written release notes will be used for stable versions only.
+  It is essential to choose the nightly that will become the release candidate as early as possible, to avoid confusion.
 - In `src/CMakeLists.txt`,
   - verify that you see `set(LEAN_VERSION_MINOR 7)` (for whichever `7` is appropriate); this should already have been updated when the development cycle began.
-  - change the `LEAN_VERSION_IS_RELEASE` line to `set(LEAN_VERSION_IS_RELEASE 1)` (this should be a change; on `master` and nightly releases it is always `0`).
+  - `set(LEAN_VERSION_IS_RELEASE 1)` (this should be a change; on `master` and nightly releases it is always `0`).
   - Commit your changes to `src/CMakeLists.txt`, and push.
 - `git tag v4.7.0-rc1`
 - `git push origin v4.7.0-rc1`
+- Ping the FRO Zulip that release notes need to be written. The release notes do not block completing the rest of this checklist.
 - Now wait, while CI runs.
-  - The CI setup parses the tag to discover the `-rc1` special description, and passes it to `cmake` using a `-D` option. The `-rc1` doesn't need to be placed in the configuration file.
   - You can monitor this at `https://github.com/leanprover/lean4/actions/workflows/ci.yml`, looking for the `v4.7.0-rc1` tag.
-  - This step can take up to two hours.
-- Verify that the release appears at https://github.com/leanprover/lean4/releases/, marked as a prerelease (this should have been done automatically by the CI release job).
-- Next we need to prepare the release notes.
-  - Run `script/release_notes.py --since v4.6.0` on the `releases/v4.7.0` branch,
-    which will report diagnostic messages on `stderr`
-    (including reporting commits that it couldn't associate with a PR, and hence will be omitted)
-    and then a chunk of markdown on `stdout`.
-    See the section "Writing the release notes" below for more information.
-  - Release notes live in https://github.com/leanprover/reference-manual, in e.g. `Manual/Releases/v4.7.0.lean`.
-    It's best if you update these at the same time as a you update the `lean-toolchain` for the `reference-manual` repository, see below. 
+  - This step can take up to an hour.
+- (GitHub release notes) Once the release appears at https://github.com/leanprover/lean4/releases/
+  - Verify that the release is marked as a prerelease (this should have been done automatically by the CI release job).
+  - In the "previous tag" dropdown, select `v4.6.0`, and click "Generate release notes".
+    This will add a list of all the commits since the last stable version.
+    - Delete "update stage0" commits, and anything with a completely inscrutable commit message.
 - Next, we will move a curated list of downstream repos to the release candidate.
   - This assumes that for each repository either:
     * There is already a *reviewed* branch `bump/v4.7.0` containing the required adaptations.
       The preparation of this branch is beyond the scope of this document.
     * The repository does not need any changes to move to the new version.
-    * Note that sometimes there are *unreviewed* but necessary changes on the `nightly-testing` branch of the repository.
-      If so, you will need to merge these into the `bump_to_v4.7.0-rc1` branch manually.
-  - For each of the repositories listed in `script/release_repos.yml`,
-    - Run `script/release_steps.py v4.7.0-rc1 <repo>` (e.g. replacing `<repo>` with `batteries`), which will walk you through the following steps:
-      - Create a new branch off `master`/`main` (as specified in the `branch` field), called `bump_to_v4.7.0-rc1`.
-      - Merge `origin/bump/v4.7.0` if relevant (i.e. `bump-branch: true` appears in `release_repos.yml`).
-      - Otherwise, you *may* need to merge `origin/nightly-testing`.
-      - Note that for `verso` and `reference-manual` development happens on `nightly-testing`, so
-        we will merge that branch into `bump_to_v4.7.0-rc1`, but it is essential in the GitHub interface that we do a rebase merge,
-        in order to preserve the history.
-      - Update the contents of `lean-toolchain` to `leanprover/lean4:v4.7.0-rc1`.
-      - In the `lakefile.toml` or `lakefile.lean`, if there are dependencies on `nightly-testing`, `bump/v4.7.0`, or specific version tags, update them to the new tag.
-        If they depend on `main` or `master`, don't change this; you've just updated the dependency, so `lake update` will take care of modifying the manifest.
-      - Run `lake update`
-      - Run `lake build && if lake check-test; then lake test; fi` to check things are working.
-      - Commit the changes as `chore: bump toolchain to v4.7.0-rc1` and push.
-      - Create a PR with title "chore: bump toolchain to v4.7.0-rc1".
-    - Merge the PR once CI completes. (Recall: for `verso` and `reference-manual` you will need to do a rebase merge.)
-    - Re-running `script/release_checklist.py` will then create the tag `v4.7.0-rc1` from `master`/`main` and push it (unless `toolchain-tag: false` in the `release_repos.yml` file)
-  - We do this for the same list of repositories as for stable releases, see above for notes about special cases.
+  - For each of the target repositories:
+    - If the repository does not need any changes (i.e. `bump/v4.7.0` does not exist) then create
+      a new PR updating `lean-toolchain` to `leanprover/lean4:v4.7.0-rc1` and running `lake update`.
+    - Otherwise:
+      - Checkout the `bump/v4.7.0` branch.
+      - Verify that the `lean-toolchain` is set to the nightly from which the release candidate was created.
+      - `git merge origin/master`
+      - Change the `lean-toolchain` to `leanprover/lean4:v4.7.0-rc1`
+      - In `lakefile.lean`, change any dependencies which were using `nightly-testing` or `bump/v4.7.0` branches
+        back to `master` or `main`, and run `lake update` for those dependencies.
+      - Run `lake build` to ensure that dependencies are found (but it's okay to stop it after a moment).
+      - `git commit`
+      - `git push`
+      - Open a PR from `bump/v4.7.0` to `master`, and either merge it yourself after CI, if appropriate,
+        or notify the maintainers that it is ready to go.
+    - Once the PR has been merged, tag `master` with `v4.7.0-rc1` and push this tag.
+  - We do this for the same list of repositories as for stable releases, see above.
     As above, there are dependencies between these, and so the process above is iterative.
     It greatly helps if you can merge the `bump/v4.7.0` PRs yourself!
-  - It is essential for Mathlib and Batteries CI that you then create the next `bump/v4.8.0` branch
+    It is essential for Mathlib CI that you then create the next `bump/v4.8.0` branch
     for the next development cycle.
     Set the `lean-toolchain` file on this branch to same `nightly` you used for this release.
-- Run `script/release_checklist.py v4.7.0-rc1` one last time to check that everything is in order.
+  - For Batteries/Aesop/Mathlib, which maintain a `nightly-testing` branch, make sure there is a tag
+    `nightly-testing-2024-02-29` with date corresponding to the nightly used for the release
+    (create it if not), and then on the `nightly-testing` branch `git reset --hard master`, and force push.
 - Make an announcement!
   This should go in https://leanprover.zulipchat.com/#narrow/stream/113486-announce, with topic `v4.7.0-rc1`.
   Please see previous announcements for suggested language.
   You will want a few bullet points for main topics from the release notes.
   Please also make sure that whoever is handling social media knows the release is out.
 - Begin the next development cycle (i.e. for `v4.8.0`) on the Lean repository, by making a PR that:
-  - Uses branch name `dev_cycle_v4.8`.
   - Updates `src/CMakeLists.txt` to say `set(LEAN_VERSION_MINOR 8)`
+  - Replaces the "release notes will be copied" text in the `v4.6.0` section of `RELEASES.md` with the
+    finalized release notes from the `releases/v4.6.0` branch.
+  - Replaces the "development in progress" in the `v4.7.0` section of `RELEASES.md` with
+    ```
+    Release candidate, release notes will be copied from the branch `releases/v4.7.0` once completed.
+    ```
+    and inserts the following section before that section:
+    ```
+    v4.8.0
+    ----------
+    Development in progress.
+    ```
+  - Removes all the entries from the `./releases_drafts/` folder.
   - Titled "chore: begin development cycle for v4.8.0"
 
+
 ## Time estimates:
 Slightly longer than the corresponding steps for a stable release.
 Similar process, but more things go wrong.
@@ -214,18 +242,15 @@ Please read https://leanprover-community.github.io/contribute/tags_and_branches.
 
 # Writing the release notes
 
-Release notes are automatically generated from the commit history, using `script/release_notes.py`.
+We are currently trying a system where release notes are compiled all at once from someone looking through the commit history.
+The exact steps are a work in progress.
+Here is the general idea:
 
-Run this as `script/release_notes.py --since v4.6.0`, where `v4.6.0` is the *previous* release version.
-This script should be run on the `releases/v4.7.0` branch.
-This will generate output for all commits since that tag.
-Note that there is output on both stderr, which should be manually reviewed,
-and on stdout, which should be manually copied into the `reference-manual` repository, in the file `Manual/Releases/v4.7.0.lean`.
+* The work is done right on the `releases/v4.6.0` branch sometime after it is created but before the stable release is made.
+  The release notes for `v4.6.0` will later be copied to `master` when we begin a new development cycle.
+* There can be material for release notes entries in commit messages.
+* There can also be pre-written entries in `./releases_drafts`, which should be all incorporated in the release notes and then deleted from the branch.
+  See `./releases_drafts/README.md` for more information.
+* The release notes should be written from a downstream expert user's point of view.
 
-The output on stderr should mostly be about commits for which the script could not find an associated PR,
-usually because a PR was rebase-merged because it contained an update to stage0.
-Some judgement is required here: ignore commits which look minor,
-but manually add items to the release notes for significant PRs that were rebase-merged.
-
-There can also be pre-written entries in `./releases_drafts`, which should be all incorporated in the release notes and then deleted from the branch.
-See `./releases_drafts/README.md` for more information.
+This section will be updated when the next release notes are written (for `v4.10.0`).

doc/do.md

@@ -0,0 +1,417 @@
+# The `do` notation
+
+Lean is a pure functional programming language, but you can write effectful code using the `do` embedded domain specific language (DSL). The following simple program prints two strings "hello" and "world" in the standard output and terminates with exit code 0. Note that the type of the program is `IO UInt32`. You can read this type as the type of values that perform input-output effects and produce a value of type `UInt32`.
+
+```lean
+def main : IO UInt32 := do
+  IO.println "hello"
+  IO.println "world"
+  return 0
+```
+The type of `IO.println` is `String → IO Unit`. That is, it is a function from `String` to `IO Unit` which indicates it may perform input-output effects and produce a value of type `Unit`. We often say that functions that may perform effects are *methods*.
+We also say a method application, such as `IO.println "hello"` is an *action*.
+Note that the examples above also demonstrates that braceless `do` blocks are whitespace sensitive.
+If you like `;`s and curly braces, you can write the example above as
+```lean
+def main : IO UInt32 := do {
+  IO.println "hello";
+  IO.println "world";
+  return 0;
+}
+```
+Semicolons can be used even when curly braces are not used. They are particularly useful when you want to "pack" more than one action in a single line.
+```lean
+def main : IO UInt32 := do
+  IO.println "hello"; IO.println "world"
+  return 0
+```
+Whitespace sensitivity in programming languages is a controversial topic
+among programmers. You should use your own style. We, the Lean developers, **love** the
+braceless and semicolon-free style.
+We believe it is clean and beautiful.
+
+The `do` DSL expands into the core Lean language. Let's inspect the different components using the commands `#print` and `#check`.
+
+```lean
+# def main : IO UInt32 := do
+#  IO.println "hello"
+#  IO.println "world"
+#  return 0
+
+#check IO.println "hello"
+-- IO Unit
+#print main
+-- Output contains the infix operator `>>=` and `pure`
+-- The following `set_option` disables notation such as `>>=` in the output
+set_option pp.notation false in
+#print main
+-- Output contains `bind` and `pure`
+#print bind
+-- bind : {m : Type u → Type v} → [self : Bind m] → {α β : Type u} →
+--        m α → (α → m β) → m β
+#print pure
+-- pure : {m : Type u → Type v} → [self : Pure m] → {α : Type u} →
+--        α → m α
+
+-- IO implements the type classes `Bind` and `Pure`.
+#check (inferInstance : Bind IO)
+#check (inferInstance : Pure IO)
+```
+The types of `bind` and `pure` may look daunting at first sight.
+They both have many implicit arguments. Let's focus first on the explicit arguments.
+`bind` has two explicit arguments `m α` and `α → m β`. The first one should
+be viewed as an action with effects `m` and producing a value of type `α`.
+The second is a function that takes a value of type `α` and produces an action
+with effects `m` and a value of type `β`. The result is `m β`. The method `bind` is composing
+these two actions. We often say `bind` is an abstract semicolon. The method `pure` converts
+a value `α` into an action that produces an action `m α`.
+
+Here is the same function being defined using `bind` and `pure` without the `do` DSL.
+```lean
+def main : IO UInt32 :=
+  bind (IO.println "hello") fun _ =>
+  bind (IO.println "world") fun _ =>
+  pure 0
+```
+
+The notations `let x <- action1; action2` and `let x ← action1; action2` are just syntax sugar for `bind action1 fun x => action2`.
+Here is a small example using it.
+```lean
+def isGreaterThan0 (x : Nat) : IO Bool := do
+  IO.println s!"value: {x}"
+  return x > 0
+
+def f (x : Nat) : IO Unit := do
+  let c <- isGreaterThan0 x
+  if c then
+    IO.println s!"{x} is greater than 0"
+  else
+    pure ()
+
+#eval f 10
+-- value: 10
+-- 10 is greater than 0
+```
+
+
+## Nested actions
+
+Note that we cannot write `if isGreaterThan0 x then ... else ...` because the condition in a `if-then-else` is a **pure** value without effects, but `isGreaterThan0 x` has type `IO Bool`. You can use the nested action notation to avoid this annoyance. Here is an equivalent definition for `f` using a nested action.
+```lean
+# def isGreaterThan0 (x : Nat) : IO Bool := do
+#  IO.println s!"x: {x}"
+#  return x > 0
+
+def f (x : Nat) : IO Unit := do
+  if (<- isGreaterThan0 x) then
+    IO.println s!"{x} is greater than 0"
+  else
+    pure ()
+
+#print f
+```
+Lean "lifts" the nested actions and introduces the `bind` for us.
+Here is an example with two nested actions. Note that both actions are executed
+even if `x = 0`.
+```lean
+# def isGreaterThan0 (x : Nat) : IO Bool := do
+#  IO.println s!"x: {x}"
+#  return x > 0
+
+def f (x y : Nat) : IO Unit := do
+  if (<- isGreaterThan0 x) && (<- isGreaterThan0 y) then
+    IO.println s!"{x} and {y} are greater than 0"
+  else
+    pure ()
+
+#eval f 0 10
+-- value: 0
+-- value: 10
+
+-- The function `f` above is equivalent to
+def g (x y : Nat) : IO Unit := do
+  let c1 <- isGreaterThan0 x
+  let c2 <- isGreaterThan0 y
+  if c1 && c2 then
+    IO.println s!"{x} and {y} are greater than 0"
+  else
+    pure ()
+
+theorem fgEqual : f = g :=
+  rfl -- proof by reflexivity
+```
+Here are two ways to achieve the short-circuit semantics in the example above
+```lean
+# def isGreaterThan0 (x : Nat) : IO Bool := do
+#  IO.println s!"x: {x}"
+#  return x > 0
+
+def f1 (x y : Nat) : IO Unit := do
+  if (<- isGreaterThan0 x <&&> isGreaterThan0 y) then
+    IO.println s!"{x} and {y} are greater than 0"
+  else
+    pure ()
+
+-- `<&&>` is the effectful version of `&&`
+-- Given `x y : IO Bool`, `x <&&> y` : m Bool`
+-- It only executes `y` if `x` returns `true`.
+
+#eval f1 0 10
+-- value: 0
+#eval f1 1 10
+-- value: 1
+-- value: 10
+-- 1 and 10 are greater than 0
+
+def f2 (x y : Nat) : IO Unit := do
+  if (<- isGreaterThan0 x) then
+    if (<- isGreaterThan0 y) then
+      IO.println s!"{x} and {y} are greater than 0"
+    else
+      pure ()
+  else
+    pure ()
+```
+
+## `if-then` notation
+
+In the `do` DSL, we can write `if c then action` as a shorthand for `if c then action else pure ()`. Here is the method `f2` using this shorthand.
+```lean
+# def isGreaterThan0 (x : Nat) : IO Bool := do
+#  IO.println s!"x: {x}"
+#  return x > 0
+
+def f2 (x y : Nat) : IO Unit := do
+  if (<- isGreaterThan0 x) then
+    if (<- isGreaterThan0 y) then
+      IO.println s!"{x} and {y} are greater than 0"
+```
+
+## Reassignments
+
+When writing effectful code, it is natural to think imperatively.
+For example, suppose we want to create an empty array `xs`,
+add `0` if some condition holds, add `1` if another condition holds,
+and then print it. In the following example, we use variable
+"shadowing" to simulate this kind of "update".
+
+```lean
+def f (b1 b2 : Bool) : IO Unit := do
+  let xs := #[]
+  let xs := if b1 then xs.push 0 else xs
+  let xs := if b2 then xs.push 1 else xs
+  IO.println xs
+
+#eval f true true
+-- #[0, 1]
+#eval f false true
+-- #[1]
+#eval f true false
+-- #[0]
+#eval f false false
+-- #[]
+```
+
+We can use tuples to simulate updates on multiple variables.
+
+```lean
+def f (b1 b2 : Bool) : IO Unit := do
+  let xs := #[]
+  let ys := #[]
+  let (xs, ys) := if b1 then (xs.push 0, ys) else (xs, ys.push 0)
+  let (xs, ys) := if b2 then (xs.push 1, ys) else (xs, ys.push 1)
+  IO.println s!"xs: {xs}, ys: {ys}"
+
+#eval f true false
+-- xs: #[0], ys: #[1]
+```
+
+We can also simulate the control-flow above using *join-points*.
+A join-point is a `let` that is always tail called and fully applied.
+The Lean compiler implements them using `goto`s.
+Here is the same example using join-points.
+
+```lean
+def f (b1 b2 : Bool) : IO Unit := do
+  let jp1 xs ys := IO.println s!"xs: {xs}, ys: {ys}"
+  let jp2 xs ys := if b2 then jp1 (xs.push 1) ys else jp1 xs (ys.push 1)
+  let xs := #[]
+  let ys := #[]
+  if b1 then jp2 (xs.push 0) ys else jp2 xs (ys.push 0)
+
+#eval f true false
+-- xs: #[0], ys: #[1]
+```
+
+You can capture complex control-flow using join-points.
+The `do` DSL offers the variable reassignment feature to make this kind of code more comfortable to write. In the following example, the `mut` modifier at `let mut xs := #[]` indicates that variable `xs` can be reassigned. The example contains two reassignments `xs := xs.push 0` and `xs := xs.push 1`. The reassignments are compiled using join-points. There is no hidden state being updated.
+
+```lean
+def f (b1 b2 : Bool) : IO Unit := do
+  let mut xs := #[]
+  if b1 then xs := xs.push 0
+  if b2 then xs := xs.push 1
+  IO.println xs
+
+#eval f true true
+-- #[0, 1]
+```
+The notation `x <- action` reassigns `x` with the value produced by the action. It is equivalent to `x := (<- action)`
+
+## Iteration
+
+The `do` DSL provides a unified notation for iterating over datastructures. Here are a few examples.
+
+```lean
+def sum (xs : Array Nat) : IO Nat := do
+  let mut s := 0
+  for x in xs do
+    IO.println s!"x: {x}"
+    s := s + x
+  return s
+
+#eval sum #[1, 2, 3]
+-- x: 1
+-- x: 2
+-- x: 3
+-- 6
+
+-- We can write pure code using the `Id.run <| do` DSL too.
+def sum' (xs : Array Nat) : Nat := Id.run <| do
+  let mut s := 0
+  for x in xs do
+    s := s + x
+  return s
+
+#eval sum' #[1, 2, 3]
+-- 6
+
+def sumEven (xs : Array Nat) : IO Nat := do
+  let mut s := 0
+  for x in xs do
+    if x % 2 == 0 then
+      IO.println s!"x: {x}"
+      s := s + x
+  return s
+
+#eval sumEven #[1, 2, 3, 6]
+-- x: 2
+-- x: 6
+-- 8
+
+def splitEvenOdd (xs : List Nat) : IO Unit := do
+  let mut evens := #[]
+  let mut odds  := #[]
+  for x in xs do
+    if x % 2 == 0 then
+      evens := evens.push x
+    else
+      odds := odds.push x
+  IO.println s!"evens: {evens}, odds: {odds}"
+
+#eval splitEvenOdd [1, 2, 3, 4]
+-- evens: #[2, 4], odds: #[1, 3]
+
+def findNatLessThan (x : Nat) (p : Nat → Bool) : IO Nat := do
+  -- [:x] is notation for the range [0, x)
+  for i in [:x] do
+    if p i then
+      return i -- `return` from the `do` block
+  throw (IO.userError "value not found")
+
+#eval findNatLessThan 10 (fun x => x > 5 && x % 4 == 0)
+-- 8
+
+def sumOddUpTo (xs : List Nat) (threshold : Nat) : IO Nat := do
+  let mut s := 0
+  for x in xs do
+    if x % 2 == 0 then
+      continue -- it behaves like the `continue` statement in imperative languages
+    IO.println s!"x: {x}"
+    s := s + x
+    if s > threshold then
+      break -- it behaves like the `break` statement in imperative languages
+  IO.println s!"result: {s}"
+  return s
+
+#eval sumOddUpTo [2, 3, 4, 11, 20, 31, 41, 51, 107] 40
+-- x: 3
+-- x: 11
+-- x: 31
+-- result: 45
+-- 45
+```
+
+TODO: describe `forIn`
+
+## Try-catch
+
+TODO
+
+## Returning early from a failed match
+
+Inside a `do` block, the pattern `let _ ← <success> | <fail>` will continue with the rest of the block if the match on the left hand side succeeds, but will execute the right hand side and exit the block on failure:
+
+```lean
+def showUserInfo (getUsername getFavoriteColor : IO (Option String)) : IO Unit := do
+  let some n ← getUsername | IO.println "no username!"
+  IO.println s!"username: {n}"
+  let some c ← getFavoriteColor | IO.println "user didn't provide a favorite color!"
+  IO.println s!"favorite color: {c}"
+
+-- username: JohnDoe
+-- favorite color: red
+#eval showUserInfo (pure <| some "JohnDoe") (pure <| some "red")
+
+-- no username
+#eval showUserInfo (pure none) (pure <| some "purple")
+
+-- username: JaneDoe
+-- user didn't provide a favorite color
+#eval showUserInfo (pure <| some "JaneDoe") (pure none)
+```
+
+## If-let
+
+Inside a `do` block, users can employ the `if let` pattern to destructure actions:
+
+```lean
+def tryIncrement (getInput : IO (Option Nat)) : IO (Except String Nat) := do
+  if let some n ← getInput
+  then return Except.ok n.succ
+  else return Except.error "argument was `none`"
+
+-- Except.ok 2
+#eval tryIncrement (pure <| some 1)
+
+-- Except.error "argument was `none`"
+#eval tryIncrement (pure <| none)
+```
+
+## Pattern matching
+
+TODO
+
+## Monads
+
+TODO
+
+## ReaderT
+
+TODO
+
+## StateT
+
+TODO
+
+## StateRefT
+
+TODO
+
+## ExceptT
+
+TODO
+
+## MonadLift and automatic lifting
+
+TODO

doc/elaborators.md

@@ -0,0 +1,8 @@
+## Elaborators
+
+TODO. See [Lean Together 2021: Metaprogramming in Lean
+4](https://youtu.be/hxQ1vvhYN_U) for an overview as well [the
+continuation](https://youtu.be/vy4JWIiiXSY) about tactic programming.
+For more information on antiquotations, see also §4.1 of [Beyond
+Notations: Hygienic Macro Expansion for Theorem Proving
+Languages](https://arxiv.org/pdf/2001.10490.pdf#page=11).

doc/enum.md

@@ -0,0 +1,190 @@
+# Enumerated Types
+
+The simplest kind of inductive type is simply a type with a finite, enumerated list of elements.
+The following command declares the enumerated type `Weekday`.
+```lean
+inductive Weekday where
+  | sunday    : Weekday
+  | monday    : Weekday
+  | tuesday   : Weekday
+  | wednesday : Weekday
+  | thursday  : Weekday
+  | friday    : Weekday
+  | saturday  : Weekday
+```
+
+The `Weekday` type has 7 constructors/elements. The constructors live in the `Weekday` namespace
+Think of `sunday`, `monday`, …, `saturday` as being distinct elements of `Weekday`,
+with no other distinguishing properties.
+```lean
+# inductive Weekday where
+#  | sunday    : Weekday
+#  | monday    : Weekday
+#  | tuesday   : Weekday
+#  | wednesday : Weekday
+#  | thursday  : Weekday
+#  | friday    : Weekday
+#  | saturday  : Weekday
+#check Weekday.sunday   -- Weekday
+#check Weekday.monday   -- Weekday
+```
+
+You can define functions by pattern matching.
+The following function converts a `Weekday` into a natural number.
+```lean
+# inductive Weekday where
+#  | sunday    : Weekday
+#  | monday    : Weekday
+#  | tuesday   : Weekday
+#  | wednesday : Weekday
+#  | thursday  : Weekday
+#  | friday    : Weekday
+#  | saturday  : Weekday
+def natOfWeekday (d : Weekday) : Nat :=
+  match d with
+  | Weekday.sunday    => 1
+  | Weekday.monday    => 2
+  | Weekday.tuesday   => 3
+  | Weekday.wednesday => 4
+  | Weekday.thursday  => 5
+  | Weekday.friday    => 6
+  | Weekday.saturday  => 7
+
+#eval natOfWeekday Weekday.tuesday -- 3
+```
+
+It is often useful to group definitions related to a type in a namespace with the same name.
+For example, we can put the function above into the ``Weekday`` namespace.
+We are then allowed to use the shorter name when we open the namespace.
+
+In the following example, we define functions from ``Weekday`` to ``Weekday`` in the namespace `Weekday`.
+```lean
+# inductive Weekday where
+#  | sunday    : Weekday
+#  | monday    : Weekday
+#  | tuesday   : Weekday
+#  | wednesday : Weekday
+#  | thursday  : Weekday
+#  | friday    : Weekday
+#  | saturday  : Weekday
+namespace Weekday
+
+def next (d : Weekday) : Weekday :=
+  match d with
+  | sunday    => monday
+  | monday    => tuesday
+  | tuesday   => wednesday
+  | wednesday => thursday
+  | thursday  => friday
+  | friday    => saturday
+  | saturday  => sunday
+
+end Weekday
+```
+It is so common to start a definition with a `match` in Lean, that Lean provides a syntax sugar for it.
+```lean
+# inductive Weekday where
+#  | sunday    : Weekday
+#  | monday    : Weekday
+#  | tuesday   : Weekday
+#  | wednesday : Weekday
+#  | thursday  : Weekday
+#  | friday    : Weekday
+#  | saturday  : Weekday
+# namespace Weekday
+def previous : Weekday -> Weekday
+  | sunday    => saturday
+  | monday    => sunday
+  | tuesday   => monday
+  | wednesday => tuesday
+  | thursday  => wednesday
+  | friday    => thursday
+  | saturday  => friday
+# end Weekday
+```
+We can use the command `#eval` to test our definitions.
+```lean
+# inductive Weekday where
+#  | sunday    : Weekday
+#  | monday    : Weekday
+#  | tuesday   : Weekday
+#  | wednesday : Weekday
+#  | thursday  : Weekday
+#  | friday    : Weekday
+#  | saturday  : Weekday
+# namespace Weekday
+# def next (d : Weekday) : Weekday :=
+#  match d with
+#  | sunday    => monday
+#  | monday    => tuesday
+#  | tuesday   => wednesday
+#  | wednesday => thursday
+#  | thursday  => friday
+#  | friday    => saturday
+#  | saturday  => sunday
+# def previous : Weekday -> Weekday
+#  | sunday    => saturday
+#  | monday    => sunday
+#  | tuesday   => monday
+#  | wednesday => tuesday
+#  | thursday  => wednesday
+#  | friday    => thursday
+#  | saturday  => friday
+def toString : Weekday -> String
+  | sunday    => "Sunday"
+  | monday    => "Monday"
+  | tuesday   => "Tuesday"
+  | wednesday => "Wednesday"
+  | thursday  => "Thursday"
+  | friday    => "Friday"
+  | saturday  => "Saturday"
+
+#eval toString (next sunday)             -- "Monday"
+#eval toString (next tuesday)            -- "Wednesday"
+#eval toString (previous wednesday)      -- "Tuesday"
+#eval toString (next (previous sunday))  -- "Sunday"
+#eval toString (next (previous monday))  -- "Monday"
+-- ..
+# end Weekday
+```
+We can now prove the general theorem that ``next (previous d) = d`` for any weekday ``d``.
+The idea is to perform a proof by cases using `match`, and rely on the fact for each constructor both
+sides of the equality reduce to the same term.
+```lean
+# inductive Weekday where
+#  | sunday    : Weekday
+#  | monday    : Weekday
+#  | tuesday   : Weekday
+#  | wednesday : Weekday
+#  | thursday  : Weekday
+#  | friday    : Weekday
+#  | saturday  : Weekday
+# namespace Weekday
+# def next (d : Weekday) : Weekday :=
+#  match d with
+#  | sunday    => monday
+#  | monday    => tuesday
+#  | tuesday   => wednesday
+#  | wednesday => thursday
+#  | thursday  => friday
+#  | friday    => saturday
+#  | saturday  => sunday
+# def previous : Weekday -> Weekday
+#  | sunday    => saturday
+#  | monday    => sunday
+#  | tuesday   => monday
+#  | wednesday => tuesday
+#  | thursday  => wednesday
+#  | friday    => thursday
+#  | saturday  => friday
+theorem nextOfPrevious (d : Weekday) : next (previous d) = d :=
+  match d with
+  | sunday    => rfl
+  | monday    => rfl
+  | tuesday   => rfl
+  | wednesday => rfl
+  | thursday  => rfl
+  | friday    => rfl
+  | saturday  => rfl
+# end Weekday
+```

doc/examples.md

@@ -0,0 +1,9 @@
+Examples
+========
+
+- [Palindromes](examples/palindromes.lean.md)
+- [Binary Search Trees](examples/bintree.lean.md)
+- [A Certified Type Checker](examples/tc.lean.md)
+- [The Well-Typed Interpreter](examples/interp.lean.md)
+- [Dependent de Bruijn Indices](examples/deBruijn.lean.md)
+- [Parametric Higher-Order Abstract Syntax](examples/phoas.lean.md)

doc/examples/ICERM2022/ctor.lean

@@ -18,7 +18,7 @@ def ctor (mvarId : MVarId) (idx : Nat) : MetaM (List MVarId) := do
     else if h : idx - 1 < ctors.length then
       mvarId.apply (.const ctors[idx - 1] us)
     else
-      throwTacticEx `ctor mvarId "invalid index, inductive datatype has only {ctors.length} constructors"
+      throwTacticEx `ctor mvarId "invalid index, inductive datatype has only {ctors.length} contructors"
 
 open Elab Tactic
 

doc/examples/ICERM2022/meta.lean

@@ -29,7 +29,7 @@ def ex3 (declName : Name) : MetaM Unit := do
     for x in xs do
       trace[Meta.debug] "{x} : {← inferType x}"
 
-def myMin [LT α] [DecidableLT α] (a b : α) : α :=
+def myMin [LT α] [DecidableRel (α := α) (·<·)] (a b : α) : α :=
   if a < b then
     a
   else

doc/examples/README.md

@@ -1,4 +0,0 @@
-These examples are checked in Lean's CI to ensure that they continue
-to work. They are included in the documentation section of the Lean
-website via a script that copies the latest version, in order to
-ensure that the website tracks Lean releases rather than `master`.

doc/examples/bintree.lean

@@ -179,7 +179,7 @@ local macro "have_eq " lhs:term:max rhs:term:max : tactic =>
   `(tactic|
     (have h : $lhs = $rhs :=
        -- TODO: replace with linarith
-       by simp +arith at *; apply Nat.le_antisymm <;> assumption
+       by simp_arith at *; apply Nat.le_antisymm <;> assumption
      try subst $lhs))
 
 /-!

doc/examples/bintree.lean.md

@@ -0,0 +1,5 @@
+(this example is rendered by Alectryon in the CI)
+
+```lean
+{{#include bintree.lean}}
+```

doc/examples/deBruijn.lean

@@ -149,7 +149,7 @@ We now define the constant folding optimization that traverses a term if replace
 /-!
 The correctness of the `Term.constFold` is proved using induction, case-analysis, and the term simplifier.
 We prove all cases but the one for `plus` using `simp [*]`. This tactic instructs the term simplifier to
-use hypotheses such as `a = b` as rewriting/simplifications rules.
+use hypotheses such as `a = b` as rewriting/simplications rules.
 We use the `split` to break the nested `match` expression in the `plus` case into two cases.
 The local variables `iha` and `ihb` are the induction hypotheses for `a` and `b`.
 The modifier `←` in a term simplifier argument instructs the term simplifier to use the equation as a rewriting rule in

doc/examples/deBruijn.lean.md

@@ -0,0 +1,5 @@
+(this example is rendered by Alectryon in the CI)
+
+```lean
+{{#include deBruijn.lean}}
+```

doc/examples/interp.lean

@@ -12,17 +12,17 @@ Remark: this example is based on an example found in the Idris manual.
 Vectors
 --------
 
-A `Vec` is a list of size `n` whose elements belong to a type `α`.
+A `Vector` is a list of size `n` whose elements belong to a type `α`.
 -/
 
-inductive Vec (α : Type u) : Nat → Type u
-  | nil : Vec α 0
-  | cons : α → Vec α n → Vec α (n+1)
+inductive Vector (α : Type u) : Nat → Type u
+  | nil : Vector α 0
+  | cons : α → Vector α n → Vector α (n+1)
 
 /-!
-We can overload the `List.cons` notation `::` and use it to create `Vec`s.
+We can overload the `List.cons` notation `::` and use it to create `Vector`s.
 -/
-infix:67 " :: " => Vec.cons
+infix:67 " :: " => Vector.cons
 
 /-!
 Now, we define the types of our simple functional language.
@@ -50,11 +50,11 @@ the builtin instance for `Add Int` as the solution.
 /-!
 Expressions are indexed by the types of the local variables, and the type of the expression itself.
 -/
-inductive HasType : Fin n → Vec Ty n → Ty → Type where
+inductive HasType : Fin n → Vector Ty n → Ty → Type where
   | stop : HasType 0 (ty :: ctx) ty
   | pop  : HasType k ctx ty → HasType k.succ (u :: ctx) ty
 
-inductive Expr : Vec Ty n → Ty → Type where
+inductive Expr : Vector Ty n → Ty → Type where
   | var   : HasType i ctx ty → Expr ctx ty
   | val   : Int → Expr ctx Ty.int
   | lam   : Expr (a :: ctx) ty → Expr ctx (Ty.fn a ty)
@@ -102,8 +102,8 @@ indexed over the types in scope. Since an environment is just another form of li
 to the vector of local variable types, we overload again the notation `::` so that we can use the usual list syntax.
 Given a proof that a variable is defined in the context, we can then produce a value from the environment.
 -/
-inductive Env : Vec Ty n → Type where
-  | nil  : Env Vec.nil
+inductive Env : Vector Ty n → Type where
+  | nil  : Env Vector.nil
   | cons : Ty.interp a → Env ctx → Env (a :: ctx)
 
 infix:67 " :: " => Env.cons

doc/examples/interp.lean.md

@@ -0,0 +1,5 @@
+(this example is rendered by Alectryon in the CI)
+
+```lean
+{{#include interp.lean}}
+```

doc/examples/palindromes.lean.md

@@ -0,0 +1,5 @@
+(this example is rendered by Alectryon in the CI)
+
+```lean
+{{#include palindromes.lean}}
+```

doc/examples/phoas.lean

@@ -225,7 +225,7 @@ We now define the constant folding optimization that traverses a term if replace
 /-!
 The correctness of the `constFold` is proved using induction, case-analysis, and the term simplifier.
 We prove all cases but the one for `plus` using `simp [*]`. This tactic instructs the term simplifier to
-use hypotheses such as `a = b` as rewriting/simplifications rules.
+use hypotheses such as `a = b` as rewriting/simplications rules.
 We use the `split` to break the nested `match` expression in the `plus` case into two cases.
 The local variables `iha` and `ihb` are the induction hypotheses for `a` and `b`.
 The modifier `←` in a term simplifier argument instructs the term simplifier to use the equation as a rewriting rule in

doc/examples/phoas.lean.md

@@ -0,0 +1,5 @@
+(this example is rendered by Alectryon in the CI)
+
+```lean
+{{#include phoas.lean}}
+```

doc/examples/tc.lean

@@ -29,7 +29,7 @@ inductive HasType : Expr → Ty → Prop
 
 /-!
 We can easily show that if `e` has type `t₁` and type `t₂`, then `t₁` and `t₂` must be equal
-by using the `cases` tactic. This tactic creates a new subgoal for every constructor,
+by using the the `cases` tactic. This tactic creates a new subgoal for every constructor,
 and automatically discharges unreachable cases. The tactic combinator `tac₁ <;> tac₂` applies
 `tac₂` to each subgoal produced by `tac₁`. Then, the tactic `rfl` is used to close all produced
 goals using reflexivity.
@@ -82,7 +82,9 @@ theorem Expr.typeCheck_correct (h₁ : HasType e ty) (h₂ : e.typeCheck ≠ .un
 /-!
 Now, we prove that if `Expr.typeCheck e` returns `Maybe.unknown`, then forall `ty`, `HasType e ty` does not hold.
 The notation `e.typeCheck` is sugar for `Expr.typeCheck e`. Lean can infer this because we explicitly said that `e` has type `Expr`.
-The proof is by induction on `e` and case analysis. Note that the tactic `simp [typeCheck]` is applied to all goal generated by the `induction` tactic, and closes
+The proof is by induction on `e` and case analysis. The tactic `rename_i` is used to to rename "inaccessible" variables.
+We say a variable is inaccessible if it is introduced by a tactic (e.g., `cases`) or has been shadowed by another variable introduced
+by the user. Note that the tactic `simp [typeCheck]` is applied to all goal generated by the `induction` tactic, and closes
 the cases corresponding to the constructors `Expr.nat` and `Expr.bool`.
 -/
 theorem Expr.typeCheck_complete {e : Expr} : e.typeCheck = .unknown → ¬ HasType e ty := by

doc/examples/tc.lean.md

@@ -0,0 +1,5 @@
+(this example is rendered by Alectryon in the CI)
+
+```lean
+{{#include tc.lean}}
+```

doc/examples/test_single.sh

@@ -1,4 +1,4 @@
 #!/usr/bin/env bash
 source ../../tests/common.sh
 
-exec_check_raw lean -Dlinter.all=false "$f"
+exec_check lean -Dlinter.all=false "$f"

doc/examples/widgets.lean

@@ -93,7 +93,7 @@ Meaning "Remote Procedure Call",this is a Lean function callable from widget cod
 Our method will take in the `name : Name` of a constant in the environment and return its type.
 By convention, we represent the input data as a `structure`.
 Since it will be sent over from JavaScript,
-we need `FromJson` and `ToJson` instance.
+we need `FromJson` and `ToJson` instnace.
 We'll see why the position field is needed later.
 -/
 

doc/examples/widgets.lean.md

@@ -0,0 +1,5 @@
+(this chapter is rendered by Alectryon in the CI)
+
+```lean
+{{#include widgets.lean}}
+```

doc/expressions.md

@@ -0,0 +1,550 @@
+Expressions
+===========
+
+Every expression in Lean has a [Type](types.md). Every type is also an
+expression of type `Sort u` for some universe level u. See [Type
+Universes](types.md#type_universes).
+
+Expression Syntax
+=================
+
+The set of expressions in Lean is defined inductively as follows:
+
+* ``Sort u`` : the universe of types at universe level ``u``
+* ``c`` : where ``c`` is an identifier denoting a declared constant or a defined object
+* ``x`` : where ``x`` is a variable in the local context in which the expression is interpreted
+* `m?`  : where `m?` is a metavariable in the metavariable context in which the expression is interpreted,
+  you can view metavariable as a "hole" that still needs to be synthesized
+* ``(x : α) → β`` : the type of functions taking an element ``x`` of ``α`` to an element of ``β``,
+  where ``β`` is an expression whose type is a ``Sort``
+* ``s t`` : the result of applying ``s`` to ``t``, where ``s`` and ``t`` are expressions
+* ``fun x : α => t`` or `λ x : α => t`: the function mapping any value ``x`` of type ``α`` to ``t``, where ``t`` is an expression
+* ``let x := t; s`` : a local definition, denotes the value of ``s`` when ``x`` is replaced by ``t``
+* `s.i`   : a projection, denotes the value of the `i`-th field of `s`
+* `lit`   : a natural number or string literal
+* `mdata k s` : the expression `s` decorated with metadata `k`, where is a key-value map
+
+Every well formed term in Lean has a *type*, which itself is an expression of type ``Sort u`` for some ``u``. The fact that a term ``t`` has type ``α`` is written ``t : α``.
+
+For an expression to be well formed, its components have to satisfy certain typing constraints. These, in turn, determine the type of the resulting term, as follows:
+
+* ``Sort u : Sort (u + 1)``
+* ``c : α``, where ``α`` is the type that ``c`` has been declared or defined to have
+* ``x : α``, where ``α`` is the type that ``x`` has been assigned in the local context where it is interpreted
+* ``?m : α``, where ``α`` is the type that ``?m`` has been declared in the metavariable context where it is interpreted
+* ``(x : α) → β : Sort (imax u v)`` where ``α : Sort u``, and ``β : Sort v`` assuming ``x : α``
+* ``s t : β[t/x]`` where ``s`` has type ``(x : α) → β`` and ``t`` has type ``α``
+* ``(fun x : α => t) : (x : α) → β`` if ``t`` has type ``β`` whenever ``x`` has type ``α``
+* ``(let x := t; s) : β[t/x]`` where ``t`` has type ``α`` and ``s`` has type ``β`` assuming ``x : α``
+* `lit : Nat` if `lit` is a numeral
+* `lit : String` if `lit` is a string literal
+* `mdata k s : α` if `s : α`
+* `s.i : α` if `s : β` and `β` is an inductive datatype with only one constructor, and `i`-th field has type `α`
+
+``Prop`` abbreviates ``Sort 0``, ``Type`` abbreviates ``Sort 1``, and
+``Type u`` abbreviates ``Sort (u + 1)`` when ``u`` is a universe
+variable. We say "``α`` is a type" to express ``α : Type u`` for some
+``u``, and we say "``p`` is a proposition" to express
+``p : Prop``. Using the *propositions as types* correspondence, given
+``p : Prop``, we refer to an expression ``t : p`` as a *proof* of ``p``. In
+contrast, given ``α : Type u`` for some ``u`` and ``t : α``, we
+sometimes refer to ``t`` as *data*.
+
+When the expression ``β`` in ``(x : α) → β`` does not depend on ``x``,
+it can be written ``α → β``. As usual, the variable ``x`` is bound in
+``(x : α) →  β``, ``fun x : α => t``, and ``let x := t; s``. The
+expression ``∀ x : α, β`` is alternative syntax for ``(x : α) →  β``,
+and is intended to be used when ``β`` is a proposition. An underscore
+can be used to generate an internal variable in a binder, as in
+``fun _ : α => t``.
+
+*Metavariables*, that is, temporary placeholders, are used in the
+process of constructing terms. Terms that are added to the
+environment contain neither metavariable nor variables, which is to
+say, they are fully elaborated and make sense in the empty context.
+
+Axioms can be declared using the ``axiom`` keyword.
+Similarly, objects can be defined in various ways, such as using ``def`` and ``theorem`` keywords.
+See [Chapter Declarations](./declarations.md) for more information.
+
+Writing an expression ``(t : α)`` forces Lean to elaborate ``t`` so that it has type ``α`` or report an error if it fails.
+
+Lean supports anonymous constructor notation, anonymous projections,
+and various forms of match syntax, including destructuring ``fun`` and
+``let``. These, as well as notation for common data types (like pairs,
+lists, and so on) are discussed in [Chapter Declarations](./declarations.md)
+in connection with inductive types.
+
+```lean
+universe u
+
+#check Sort 0
+#check Prop
+#check Sort 1
+#check Type
+#check Sort u
+#check Sort (u+1)
+
+#check Nat → Bool
+#check (α : Type u) → List α
+#check (α : Type u) → (β : Type u) → Sum α β
+#check fun x : Nat => x
+#check fun (α : Type u) (x : α) => x
+#check let x := 5; x * 2
+#check "hello"
+#check (fun x => x) true
+```
+
+Implicit Arguments
+==================
+
+When declaring arguments to defined objects in Lean (for example, with
+``def``, ``theorem``, ``axiom``, ``constant``, ``inductive``, or
+``structure``; see [Chapter Declarations](./declarations.md) or when
+declaring variables in sections (see [Other Commands](./other_commands.md)),
+arguments can be annotated as *explicit* or *implicit*.
+This determines how expressions containing the object are interpreted.
+
+* ``(x : α)`` : an explicit argument of type ``α``
+* ``{x : α}`` : an implicit argument, eagerly inserted
+* ``⦃x : α⦄`` or ``{{x : α}}`` : an implicit argument, weakly inserted
+* ``[x : α]`` : an implicit argument that should be inferred by type class resolution
+* ``(x : α := v)`` : an optional argument, with default value ``v``
+* ``(x : α := by tac)`` : an implicit argument, to be synthesized by tactic ``tac``
+
+The name of the variable can be omitted from a class resolution
+argument, in which case an internal name is generated.
+
+When a function has an explicit argument, you can nonetheless ask
+Lean's elaborator to infer the argument automatically, by entering it
+as an underscore (``_``). Conversely, writing ``@foo`` indicates that
+all of the arguments to be ``foo`` are to be given explicitly,
+independent of how ``foo`` was declared.  You can also provide a value
+for an implicit parameter using named arguments.  Named arguments
+enable you to specify an argument for a parameter by matching the
+argument with its name rather than with its position in the parameter
+list.  If you don't remember the order of the parameters but know
+their names, you can send the arguments in any order.  You may also
+provide the value for an implicit parameter whenLean failed to infer
+it. Named arguments also improve the readability of your code by
+identifying what each argument represents.
+
+
+```lean
+def add (x y : Nat) : Nat :=
+  x + y
+
+#check add 2 3 -- Nat
+#eval add 2 3  -- 5
+
+def id1 (α : Type u) (x : α) : α := x
+
+#check id1 Nat 3
+#check id1 _ 3
+
+def id2 {α : Type u} (x : α) : α := x
+
+#check id2 3
+#check @id2 Nat 3
+#check id2 (α := Nat) 3
+#check id2
+#check id2 (α := Nat)
+
+def id3 {{α : Type u}} (x : α) : α := x
+
+#check id3 3
+#check @id3 Nat 3
+#check (id3 : (α : Type) → α → α)
+
+class Cls where
+  val : Nat
+
+instance Cls_five : Cls where
+  val := 5
+
+def ex2 [c : Cls] : Nat := c.val
+
+example : ex2 = 5 := rfl
+
+def ex2a [Cls] : Nat := ex2
+
+example : ex2a = 5 := rfl
+
+def ex3 (x : Nat := 5) := x
+
+#check ex3 2
+#check ex3
+
+example : ex3 = 5 := rfl
+
+def ex4 (x : Nat) (y : Nat := x) : Nat :=
+  x * y
+
+example : ex4 x = x * x :=
+  rfl
+```
+
+Basic Data Types and Assertions
+===============================
+
+The core library contains a number of basic data types, such as the
+natural numbers (`Nat`), the integers (`Int`), the
+booleans (``Bool``), and common operations on these, as well as the
+usual logical quantifiers and connectives. Some example are given
+below. A list of common notations and their precedences can be found
+in a [file](https://github.com/leanprover/lean4/blob/master/src/Init/Notation.lean)
+in the core library. The core library also contains a number of basic
+data type constructors. Definitions can also be found the
+[Data](https://github.com/leanprover/lean4/blob/master/src/Init/Data)
+directory of the core library. For more information, see also [Chapter libraries](./libraries.md).
+
+```
+/- numbers -/
+def f1 (a b c : Nat) : Nat :=
+  a^2 + b^2 + c^2
+
+def p1 (a b c d : Nat) : Prop :=
+  (a + b)^c ≤ d
+
+def p2 (i j k : Int) : Prop :=
+  i % (j * k) = 0
+
+
+/- booleans -/
+
+def f2 (a b c : Bool) : Bool :=
+  a && (b || c)
+
+/- pairs -/
+
+#eval (1, 2)
+
+def p : Nat × Bool := (1, false)
+
+section
+variable (a b c : Nat) (p : Nat × bool)
+
+#check (1, 2)
+#check p.1 * 2
+#check p.2 && tt
+#check ((1, 2, 3) : Nat × Nat × Nat)
+end
+
+/- lists -/
+section
+variable x y z : Nat
+variable xs ys zs : list Nat
+open list
+
+#check (1 :: xs) ++ (y :: zs) ++ [1,2,3]
+#check append (cons 1 xs) (cons y zs)
+#check map (λ x, x^2) [1, 2, 3]
+end
+
+/- sets -/
+section
+variable s t u : set Nat
+
+#check ({1, 2, 3} ∩ s) ∪ ({x | x < 7} ∩ t)
+end
+
+/- strings and characters -/
+#check "hello world"
+#check 'a'
+
+/- assertions -/
+#check ∀ a b c n : Nat,
+  a ≠ 0 ∧ b ≠ 0 ∧ c ≠ 0 ∧ n > 2 → a^n + b^n ≠ c^n
+
+def unbounded (f : Nat → Nat) : Prop := ∀ M, ∃ n, f n ≥ M
+```
+.. _constructors_projections_and_matching:
+
+Constructors, Projections, and Matching
+=======================================
+
+Lean's foundation, the *Calculus of Inductive Constructions*, supports the declaration of *inductive types*. Such types can have any number of *constructors*, and an associated *eliminator* (or *recursor*). Inductive types with one constructor, known as *structures*, have *projections*. The full syntax of inductive types is described in [Declarations](declarations.md), but here we describe some syntactic elements that facilitate their use in expressions.
+
+When Lean can infer the type of an expression and it is an inductive type with one constructor, then one can write ``⟨a1, a2, ..., an⟩`` to apply the constructor without naming it. For example, ``⟨a, b⟩`` denotes ``prod.mk a b`` in a context where the expression can be inferred to be a pair, and ``⟨h₁, h₂⟩`` denotes ``and.intro h₁ h₂`` in a context when the expression can be inferred to be a conjunction. The notation will nest constructions automatically, so ``⟨a1, a2, a3⟩`` is interpreted as ``prod.mk a1 (prod.mk a2 a3)`` when the expression is expected to have a type of the form ``α1 × α2 × α3``. (The latter is interpreted as ``α1 × (α2 × α3)``, since the product associates to the right.)
+
+Similarly, one can use "dot notation" for projections: one can write ``p.fst`` and ``p.snd`` for ``prod.fst p`` and ``prod.snd p`` when Lean can infer that ``p`` is an element of a product, and ``h.left`` and ``h.right`` for ``and.left h`` and ``and.right h`` when ``h`` is a conjunction.
+
+The anonymous projector notation can used more generally for any objects defined in a *namespace* (see [Other Commands](other_commands.md)). For example, if ``l`` has type ``list α`` then ``l.map f`` abbreviates ``list.map f l``, in which ``l`` has been placed at the first argument position where ``list.map`` expects a ``list``.
+
+Finally, for data types with one constructor, one destruct an element by pattern matching using the ``let`` and ``assume`` constructs, as in the examples below. Internally, these are interpreted using the ``match`` construct, which is in turn compiled down for the eliminator for the inductive type, as described in [Declarations](declarations.md).
+
+.. code-block:: lean
+
+    universes u v
+    variable {α : Type u} {β : Type v}
+
+    def p : Nat × ℤ := ⟨1, 2⟩
+    #check p.fst
+    #check p.snd
+
+    def p' : Nat × ℤ × bool := ⟨1, 2, tt⟩
+    #check p'.fst
+    #check p'.snd.fst
+    #check p'.snd.snd
+
+    def swap_pair (p : α × β) : β × α :=
+    ⟨p.snd, p.fst⟩
+
+    theorem swap_conj {a b : Prop} (h : a ∧ b) : b ∧ a :=
+    ⟨h.right, h.left⟩
+
+    #check [1, 2, 3].append [2, 3, 4]
+    #check [1, 2, 3].map (λ x, x^2)
+
+    example (p q : Prop) : p ∧ q → q ∧ p :=
+    λ h, ⟨h.right, h.left⟩
+
+    def swap_pair' (p : α × β) : β × α :=
+    let (x, y) := p in (y, x)
+
+    theorem swap_conj' {a b : Prop} (h : a ∧ b) : b ∧ a :=
+    let ⟨ha, hb⟩ := h in ⟨hb, ha⟩
+
+    def swap_pair'' : α × β → β × α :=
+    λ ⟨x, y⟩, (y, x)
+
+    theorem swap_conj'' {a b : Prop} : a ∧ b → b ∧ a :=
+    assume ⟨ha, hb⟩, ⟨hb, ha⟩
+
+Structured Proofs
+=================
+
+Syntactic sugar is provided for writing structured proof terms:
+
+* ``have h : p := s; t`` is sugar for ``(fun h : p => t) s``
+* ``suffices h : p from s; t`` is sugar for ``(λ h : p => s) t``
+* ``suffices h : p by s; t`` is sugar for ``(suffixes h : p from by s; t)``
+* ``show p from t`` is sugar for ``(have this : p := t; this)``
+* ``show p by tac`` is sugar for ``(show p from by tac)``
+
+Types can be omitted when they can be inferred by Lean. Lean also
+allows ``have : p := t; s``, which gives the assumption the
+name ``this`` in the local context.  Similarly, Lean recognizes the
+variant ``suffices p from s; t``, which use the name ``this`` for the new hypothesis.
+
+The notation ``‹p›`` is notation for ``(by assumption : p)``, and can
+therefore be used to apply hypotheses in the local context.
+
+As noted in [Constructors, Projections and Matching](#constructors_projections_and_matching),
+anonymous constructors and projections and match syntax can be used in proofs just as in expressions that denote data.
+
+.. code-block:: lean
+
+    example (p q r : Prop) : p → (q ∧ r) → p ∧ q :=
+    assume h₁ : p,
+    assume h₂ : q ∧ r,
+    have h₃ : q, from and.left h₂,
+    show p ∧ q, from and.intro h₁ h₃
+
+    example (p q r : Prop) : p → (q ∧ r) → p ∧ q :=
+    assume : p,
+    assume : q ∧ r,
+    have q, from and.left this,
+    show p ∧ q, from and.intro ‹p› this
+
+    example (p q r : Prop) : p → (q ∧ r) → p ∧ q :=
+    assume h₁ : p,
+    assume h₂ : q ∧ r,
+    suffices h₃ : q, from and.intro h₁ h₃,
+    show q, from and.left h₂
+
+Lean also supports a calculational environment, which is introduced with the keyword ``calc``. The syntax is as follows:
+
+.. code-block:: text
+
+    calc
+      <expr>_0  'op_1'  <expr>_1  ':'  <proof>_1
+        '...'   'op_2'  <expr>_2  ':'  <proof>_2
+         ...
+        '...'   'op_n'  <expr>_n  ':'  <proof>_n
+
+Each ``<proof>_i`` is a proof for ``<expr>_{i-1} op_i <expr>_i``.
+
+Here is an example:
+
+.. code-block:: lean
+
+    variable (a b c d e : Nat)
+    variable h1 : a = b
+    variable h2 : b = c + 1
+    variable h3 : c = d
+    variable h4 : e = 1 + d
+
+    theorem T : a = e :=
+    calc
+      a     = b      : h1
+        ... = c + 1  : h2
+        ... = d + 1  : congr_arg _ h3
+        ... = 1 + d  : add_comm d (1 : Nat)
+        ... =  e     : eq.symm h4
+
+The style of writing proofs is most effective when it is used in conjunction with the ``simp`` and ``rewrite`` tactics.
+
+.. _computation:
+
+Computation
+===========
+
+Two expressions that differ up to a renaming of their bound variables are said to be *α-equivalent*, and are treated as syntactically equivalent by Lean.
+
+Every expression in Lean has a natural computational interpretation, unless it involves classical elements that block computation, as described in the next section. The system recognizes the following notions of *reduction*:
+
+* *β-reduction* : An expression ``(λ x, t) s`` β-reduces to ``t[s/x]``, that is, the result of replacing ``x`` by ``s`` in ``t``.
+* *ζ-reduction* : An expression ``let x := s in t`` ζ-reduces to ``t[s/x]``.
+* *δ-reduction* : If ``c`` is a defined constant with definition ``t``, then ``c`` δ-reduces to to ``t``.
+* *ι-reduction* : When a function defined by recursion on an inductive type is applied to an element given by an explicit constructor, the result ι-reduces to the specified function value, as described in [Inductive Types](inductive.md).
+
+The reduction relation is transitive, which is to say, is ``s`` reduces to ``s'`` and ``t`` reduces to ``t'``, then ``s t`` reduces to ``s' t'``, ``λ x, s`` reduces to ``λ x, s'``, and so on. If ``s`` and ``t`` reduce to a common term, they are said to be *definitionally equal*. Definitional equality is defined to be the smallest equivalence relation that satisfies all these properties and also includes α-equivalence and the following two relations:
+
+* *η-equivalence* : An expression ``(λx, t x)`` is η-equivalent to ``t``, assuming ``x`` does not occur in ``t``.
+* *proof irrelevance* : If ``p : Prop``, ``s : p``, and ``t : p``, then ``s`` and ``t`` are  considered to be equivalent.
+
+This last fact reflects the intuition that once we have proved a proposition ``p``, we only care that is has been proved; the proof does nothing more than witness the fact that ``p`` is true.
+
+Definitional equality is a strong notion of equality of values. Lean's logical foundations sanction treating definitionally equal terms as being the same when checking that a term is well-typed and/or that it has a given type.
+
+The reduction relation is believed to be strongly normalizing, which is to say, every sequence of reductions applied to a term will eventually terminate. The property guarantees that Lean's type-checking algorithm terminates, at least in principle. The consistency of Lean and its soundness with respect to set-theoretic semantics do not depend on either of these properties.
+
+Lean provides two commands to compute with expressions:
+
+* ``#reduce t`` : use the kernel type-checking procedures to carry out reductions on ``t`` until no more reductions are possible, and show the result
+* ``#eval t`` : evaluate ``t`` using a fast bytecode evaluator, and show the result
+
+Every computable definition in Lean is compiled to bytecode at definition time. Bytecode evaluation is more liberal than kernel evaluation: types and all propositional information are erased, and functions are evaluated using a stack-based virtual machine. As a result, ``#eval`` is more efficient than ``#reduce,`` and can be used to execute complex programs. In contrast, ``#reduce`` is designed to be small and reliable, and to produce type-correct terms at each step. Bytecode is never used in type checking, so as far as soundness and consistency are concerned, only kernel reduction is part of the trusted computing base.
+
+.. code-block:: lean
+
+    #reduce (fun x => x + 3) 5
+    #eval   (fun x => x + 3) 5
+
+    #reduce let x := 5; x + 3
+    #eval   let x := 5; x + 3
+
+    def f x := x + 3
+
+    #reduce f 5
+    #eval   f 5
+
+    #reduce @Nat.rec (λ n => Nat) (0 : Nat)
+                     (λ n recval : Nat => recval + n + 1) (5 : Nat)
+
+    def g : Nat → Nat
+    | 0     => 0
+    | (n+1) => g n + n + 1
+
+    #reduce g 5
+    #eval   g 5
+
+    #eval   g 5000
+
+    example : (fun x => x + 3) 5 = 8 := rfl
+    example : (fun x => f x) = f := rfl
+    example (p : Prop) (h₁ h₂ : p) : h₁ = h₂ := rfl
+
+Note: the combination of proof irrelevance and singleton ``Prop`` elimination in ι-reduction renders the ideal version of definitional equality, as described above, undecidable. Lean's procedure for checking definitional equality is only an approximation to the ideal. It is not transitive, as illustrated by the example below. Once again, this does not compromise the consistency or soundness of Lean; it only means that Lean is more conservative in the terms it recognizes as well typed, and this does not cause problems in practice. Singleton elimination will be discussed in greater detail in [Inductive Types](inductive.md).
+
+.. code-block:: lean
+
+    def R (x y : unit) := false
+    def accrec := @acc.rec unit R (λ_, unit) (λ _ a ih, ()) ()
+    example (h) : accrec h = accrec (acc.intro _ (λ y, acc.inv h)) :=
+                  rfl
+    example (h) : accrec (acc.intro _ (λ y, acc.inv h)) = () := rfl
+    example (h) : accrec h = () := sorry   -- rfl fails
+
+
+Axioms
+======
+
+Lean's foundational framework consists of:
+
+- type universes and dependent function types, as described above
+
+- inductive definitions, as described in [Inductive Types](inductive.md) and
+[Inductive Families](declarations.md#inductive-families).
+
+In addition, the core library defines (and trusts) the following axiomatic extensions:
+
+- propositional extensionality:
+
+  .. code-block:: lean
+
+     namespace hide
+
+     -- BEGIN
+     axiom propext {a b : Prop} : (a ↔ b) → a = b
+     -- END
+
+     end hide
+
+- quotients:
+
+  .. code-block:: lean
+
+     namespace hide
+     -- BEGIN
+     universes u v
+
+     constant quot      : Π {α : Sort u}, (α → α → Prop) → Sort u
+
+     constant quot.mk   : Π {α : Sort u} (r : α → α → Prop),
+                          α → quot r
+
+     axiom    quot.ind  : ∀ {α : Sort u} {r : α → α → Prop}
+                            {β : quot r → Prop},
+                          (∀ a, β (quot.mk r a)) →
+                            ∀ (q : quot r), β q
+
+     constant quot.lift : Π {α : Sort u} {r : α → α → Prop}
+                            {β : Sort u} (f : α → β),
+                          (∀ a b, r a b → f a = f b) → quot r → β
+
+     axiom quot.sound   : ∀ {α : Type u} {r : α → α → Prop}
+                            {a b : α},
+                          r a b → quot.mk r a = quot.mk r b
+     -- END
+     end hide
+
+  ``quot r`` represents the quotient of ``α`` by the smallest equivalence relation containing ``r``. ``quot.mk`` and ``quot.lift`` satisfy the following computation rule:
+
+  .. code-block:: text
+
+     quot.lift f h (quot.mk r a) = f a
+
+- choice:
+
+  .. code-block:: lean
+
+     namespace hide
+     universe u
+
+     -- BEGIN
+     axiom choice {α : Sort u} : nonempty α → α
+     -- END
+
+     end hide
+
+  Here ``nonempty α`` is defined as follows:
+
+  .. code-block:: lean
+
+     namespace hide
+     universe u
+
+     -- BEGIN
+     class inductive nonempty (α : Sort u) : Prop
+     | intro : α → nonempty
+     -- END
+
+     end hide
+
+  It is equivalent to  ``∃ x : α, true``.
+
+The quotient construction implies function extensionality. The ``choice`` principle, in conjunction with the others, makes the axiomatic foundation classical; in particular, it implies the law of the excluded middle and propositional decidability. Functions that make use of ``choice`` to produce data are incompatible with a computational interpretation, and do not produce bytecode. They have to be declared ``noncomputable``.
+
+For metaprogramming purposes, Lean also allows the definition of objects which stand outside the object language. These are denoted with the ``meta`` keyword, as described in [Metaprogramming](metaprogramming.md).

doc/faq.md

@@ -0,0 +1,55 @@
+Frequently Asked Questions
+==========================
+
+### What is Lean?
+
+Lean is a new open source theorem prover being developed at Microsoft Research.
+It is a research project that aims to bridge the gap between interactive and automated theorem proving.
+Lean can be also used as a programming language. Actually, some Lean features are implemented in Lean itself.
+
+### Should I use Lean?
+
+Lean is under heavy development, and we are constantly trying new
+ideas and tweaking the system.  It is a research project and not a product.
+Things change rapidly, and we constantly break backward compatibility.
+Lean comes "as is", you should not expect we will fix bugs and/or add new features for your project.
+We have our own priorities, and will not change them to accommodate your needs.
+Even if you implement a new feature or fix a bug, we may not want to merge it because
+it may conflict with our plans for Lean, it may not be performant, we may not want to maintain it,
+we may be busy, etc. If you really need this new feature or bug fix, we suggest you create your own fork and maintain it yourself.
+
+### Where is the documentation?
+
+This is the Lean 4 manual. It is a work in progress, but it will eventually cover the whole language.
+A public and very active chat room dedicated to Lean is open on [Zulip](https://leanprover.zulipchat.com).
+It is a good place to interact with other Lean users.
+
+### Should I use Lean to teach a course?
+
+Lean has been used to teach courses on logic, type theory and programming languages at CMU and the University of Washington.
+The lecture notes for the CMU course [Logic and Proof](https://lean-lang.org/logic_and_proof) are available online,
+but they are for Lean 3.
+If you decide to teach a course using Lean, we suggest you prepare all material before the beginning of the course, and
+make sure that Lean attends all your needs. You should not expect we will fix bugs and/or add features needed for your course.
+
+### Are there IDEs for Lean?
+
+Yes, see [Setting Up Lean](./setup.md).
+
+### Is Lean sound? How big is the kernel? Should I trust it?
+
+Lean has a relatively small kernel.
+Several independent checkers have been implemented for Lean 3. Two of them are
+[tc](https://github.com/leanprover/tc) and [trepplein](https://github.com/gebner/trepplein).
+We expect similar independent checkers will be built for Lean 4.
+
+### Should I open a new issue?
+
+We use [GitHub](https://github.com/leanprover/lean4/issues) to track bugs and new features.
+Bug reports are always welcome, but nitpicking issues are not (e.g., the error message is confusing).
+See also our [contribution guidelines](https://github.com/leanprover/lean4/blob/master/CONTRIBUTING.md).
+
+### Is it Lean, LEAN, or L∃∀N?
+
+We always use "Lean" in writing.
+When specifying a major version number, we append it together with a single space: Lean 4.

doc/flake.lock

@@ -0,0 +1,151 @@
+{
+  "nodes": {
+    "alectryon": {
+      "flake": false,
+      "locked": {
+        "lastModified": 1654613606,
+        "narHash": "sha256-IGCn1PzTyw8rrwmyWUiw3Jo/dyZVGkMslnHYW7YB8yk=",
+        "owner": "Kha",
+        "repo": "alectryon",
+        "rev": "c3b16f650665745e1da4ddfcc048d3bd639f71d5",
+        "type": "github"
+      },
+      "original": {
+        "owner": "Kha",
+        "ref": "typeid",
+        "repo": "alectryon",
+        "type": "github"
+      }
+    },
+    "flake-utils": {
+      "inputs": {
+        "systems": "systems"
+      },
+      "locked": {
+        "lastModified": 1710146030,
+        "narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=",
+        "owner": "numtide",
+        "repo": "flake-utils",
+        "rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a",
+        "type": "github"
+      },
+      "original": {
+        "owner": "numtide",
+        "repo": "flake-utils",
+        "type": "github"
+      }
+    },
+    "lean": {
+      "inputs": {
+        "flake-utils": "flake-utils",
+        "nixpkgs": "nixpkgs",
+        "nixpkgs-old": "nixpkgs-old"
+      },
+      "locked": {
+        "lastModified": 0,
+        "narHash": "sha256-saRAtQ6VautVXKDw1XH35qwP0KEBKTKZbg/TRa4N9Vw=",
+        "path": "../.",
+        "type": "path"
+      },
+      "original": {
+        "path": "../.",
+        "type": "path"
+      }
+    },
+    "leanInk": {
+      "flake": false,
+      "locked": {
+        "lastModified": 1704976501,
+        "narHash": "sha256-FSBUsbX0HxakSnYRYzRBDN2YKmH9EkA0q9p7TSPEJTI=",
+        "owner": "leanprover",
+        "repo": "LeanInk",
+        "rev": "51821e3c2c032c88e4b2956483899d373ec090c4",
+        "type": "github"
+      },
+      "original": {
+        "owner": "leanprover",
+        "ref": "refs/pull/57/merge",
+        "repo": "LeanInk",
+        "type": "github"
+      }
+    },
+    "mdBook": {
+      "flake": false,
+      "locked": {
+        "lastModified": 1660074464,
+        "narHash": "sha256-W30G7AeWBjdJE/CQZJU5vJjaDGZtpmxEKNMEvaYtuF8=",
+        "owner": "leanprover",
+        "repo": "mdBook",
+        "rev": "9321c10c502cd59eea8afc4325a84eab3ddf9391",
+        "type": "github"
+      },
+      "original": {
+        "owner": "leanprover",
+        "repo": "mdBook",
+        "type": "github"
+      }
+    },
+    "nixpkgs": {
+      "locked": {
+        "lastModified": 1710889954,
+        "narHash": "sha256-Pr6F5Pmd7JnNEMHHmspZ0qVqIBVxyZ13ik1pJtm2QXk=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "7872526e9c5332274ea5932a0c3270d6e4724f3b",
+        "type": "github"
+      },
+      "original": {
+        "owner": "NixOS",
+        "ref": "nixpkgs-unstable",
+        "repo": "nixpkgs",
+        "type": "github"
+      }
+    },
+    "nixpkgs-old": {
+      "flake": false,
+      "locked": {
+        "lastModified": 1581379743,
+        "narHash": "sha256-i1XCn9rKuLjvCdu2UeXKzGLF6IuQePQKFt4hEKRU5oc=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "34c7eb7545d155cc5b6f499b23a7cb1c96ab4d59",
+        "type": "github"
+      },
+      "original": {
+        "owner": "NixOS",
+        "ref": "nixos-19.03",
+        "repo": "nixpkgs",
+        "type": "github"
+      }
+    },
+    "root": {
+      "inputs": {
+        "alectryon": "alectryon",
+        "flake-utils": [
+          "lean",
+          "flake-utils"
+        ],
+        "lean": "lean",
+        "leanInk": "leanInk",
+        "mdBook": "mdBook"
+      }
+    },
+    "systems": {
+      "locked": {
+        "lastModified": 1681028828,
+        "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
+        "owner": "nix-systems",
+        "repo": "default",
+        "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-systems",
+        "repo": "default",
+        "type": "github"
+      }
+    }
+  },
+  "root": "root",
+  "version": 7
+}

doc/flake.nix

@@ -0,0 +1,94 @@
+{
+  description = "Lean documentation";
+
+  inputs.lean.url = path:../.;
+  inputs.flake-utils.follows = "lean/flake-utils";
+  inputs.mdBook = {
+    url = "github:leanprover/mdBook";
+    flake = false;
+  };
+  inputs.alectryon = {
+    url = "github:Kha/alectryon/typeid";
+    flake = false;
+  };
+  inputs.leanInk = {
+    url = "github:leanprover/LeanInk/refs/pull/57/merge";
+    flake = false;
+  };
+
+  outputs = inputs@{ self, ... }: inputs.flake-utils.lib.eachDefaultSystem (system:
+    with inputs.lean.packages.${system}.deprecated; with nixpkgs;
+    let
+      doc-src = lib.sourceByRegex ../. ["doc.*" "tests(/lean(/beginEndAsMacro.lean)?)?"];
+    in {
+    packages = rec {
+      lean-mdbook = mdbook.overrideAttrs (drv: rec {
+        name = "lean-${mdbook.name}";
+        src = inputs.mdBook;
+        cargoDeps = drv.cargoDeps.overrideAttrs (_: {
+          inherit src;
+          outputHash = "sha256-CO3A9Kpp4sIvkT9X3p+GTidazk7Fn4jf0AP2PINN44A=";
+        });
+        doCheck = false;
+      });
+      book = stdenv.mkDerivation {
+        name ="lean-doc";
+        src = doc-src;
+        buildInputs = [ lean-mdbook ];
+        buildCommand = ''
+          mkdir $out
+          # necessary for `additional-css`...?
+          cp -r --no-preserve=mode $src/doc/* .
+          # overwrite stub .lean.md files
+          cp -r ${inked}/* .
+          mdbook build -d $out
+        '';
+      };
+      leanInk = (buildLeanPackage {
+        name = "Main";
+        src = inputs.leanInk;
+        deps = [ (buildLeanPackage {
+          name = "LeanInk";
+          src = inputs.leanInk;
+        }) ];
+        executableName = "leanInk";
+        linkFlags = ["-rdynamic"];
+      }).executable;
+      alectryon = python3Packages.buildPythonApplication {
+        name = "alectryon";
+        src = inputs.alectryon;
+        propagatedBuildInputs =
+          [ leanInk lean-all ] ++
+          # https://github.com/cpitclaudel/alectryon/blob/master/setup.cfg
+          (with python3Packages; [ pygments dominate beautifulsoup4 docutils ]);
+        doCheck = false;
+      };
+      renderLeanMod = mod: mod.overrideAttrs (final: prev: {
+        name = "${prev.name}.md";
+        buildInputs = prev.buildInputs ++ [ alectryon ];
+        outputs = [ "out" ];
+        buildCommand = ''
+          dir=$(dirname $relpath)
+          mkdir -p $dir out/$dir
+          if [ -d $src ]; then cp -r $src/. $dir/; else cp $src $leanPath; fi
+          alectryon --frontend lean4+markup $leanPath --backend webpage -o $out/$leanPath.md
+        '';
+      });
+      renderPackage = pkg: symlinkJoin {
+        name = "${pkg.name}-mds";
+        paths = map renderLeanMod (lib.attrValues pkg.mods);
+      };
+      literate = buildLeanPackage {
+        name = "literate";
+        src = ./.;
+        roots = [
+          { mod = "examples"; glob = "submodules"; }
+          { mod = "monads"; glob = "submodules"; }
+        ];
+      };
+      inked = renderPackage literate;
+      doc = book;
+    };
+    defaultPackage = self.packages.${system}.doc;
+  });
+}

doc/float.md

@@ -0,0 +1 @@
+# Float

doc/fplean.md

@@ -0,0 +1,7 @@
+Functional Programming in Lean
+=======================
+
+The goal of [this book](https://lean-lang.org/functional_programming_in_lean/) is to be an accessible introduction to using Lean 4 as a programming language.
+It should be useful both to people who want to use Lean as a general-purpose programming language and to mathematicians who want to develop larger-scale proof automation but do not have a background in functional programming.
+It does not assume any background with functional programming, though it's probably not a good first book on programming in general.
+New content will be added once per month until it's done.

doc/funabst.md

@@ -0,0 +1,145 @@
+## Function Abstraction and Evaluation
+
+We have seen that if we have ``m n : Nat``, then we have ``(m, n) : Nat × Nat``.
+This gives us a way of creating pairs of natural numbers.
+Conversely, if we have ``p : Nat × Nat``, then
+we have ``p.1 : Nat`` and ``p.2 : Nat``.
+This gives us a way of "using" a pair, by extracting its two components.
+
+We already know how to "use" a function ``f : α → β``, namely,
+we can apply it to an element ``a : α`` to obtain ``f a : β``.
+But how do we create a function from another expression?
+
+The companion to application is a process known as "lambda abstraction."
+Suppose that giving a variable ``x : α`` we can construct an expression ``t : β``.
+Then the expression ``fun (x : α) => t``, or, equivalently, ``λ (x : α) => t``, is an object of type ``α → β``.
+Think of this as the function from ``α`` to ``β`` which maps any value ``x`` to the value ``t``,
+which may depend on ``x``.
+
+```lean
+#check fun (x : Nat) => x + 5
+#check λ (x : Nat) => x + 5
+#check fun x : Nat => x + 5
+#check λ x : Nat => x + 5
+```
+
+Here are some more examples:
+
+```lean
+constant f : Nat → Nat
+constant h : Nat → Bool → Nat
+
+#check fun x : Nat => fun y : Bool => h (f x) y   -- Nat → Bool → Nat
+#check fun (x : Nat) (y : Bool) => h (f x) y      -- Nat → Bool → Nat
+#check fun x y => h (f x) y                       -- Nat → Bool → Nat
+```
+
+Lean interprets the final three examples as the same expression; in the last expression,
+Lean infers the type of ``x`` and ``y`` from the types of ``f`` and ``h``.
+
+Some mathematically common examples of operations of functions can be described in terms of lambda abstraction:
+
+```lean
+constant f : Nat → String
+constant g : String → Bool
+constant b : Bool
+
+#check fun x : Nat => x        -- Nat → Nat
+#check fun x : Nat => b        -- Nat → Bool
+#check fun x : Nat => g (f x)  -- Nat → Bool
+#check fun x => g (f x)        -- Nat → Bool
+```
+
+Think about what these expressions mean. The expression ``fun x : Nat => x`` denotes the identity function on ``Nat``,
+the expression ``fun x : α => b`` denotes the constant function that always returns ``b``,
+and ``fun x : Nat => g (f x)``, denotes the composition of ``f`` and ``g``.
+We can, in general, leave off the type annotation on a variable and let Lean infer it for us.
+So, for example, we can write ``fun x => g (f x)`` instead of ``fun x : Nat => g (f x)``.
+
+We can abstract over the constants `f` and `g` in the previous definitions:
+
+```lean
+#check fun (g : String → Bool) (f : Nat → String) (x : Nat) => g (f x)
+-- (String → Bool) → (Nat → String) → Nat → Bool
+```
+
+We can also abstract over types:
+
+```lean
+#check fun (α β γ : Type) (g : β → γ) (f : α → β) (x : α) => g (f x)
+```
+The last expression, for example, denotes the function that takes three types, ``α``, ``β``, and ``γ``, and two functions, ``g : β → γ`` and ``f : α → β``, and returns the composition of ``g`` and ``f``. (Making sense of the type of this function requires an understanding of dependent products, which we will explain below.) Within a lambda expression ``fun x : α => t``, the variable ``x`` is a "bound variable": it is really a placeholder, whose "scope" does not extend beyond ``t``.
+For example, the variable ``b`` in the expression ``fun (b : β) (x : α) => b`` has nothing to do with the constant ``b`` declared earlier.
+In fact, the expression denotes the same function as ``fun (u : β) (z : α), u``. Formally, the expressions that are the same up to a renaming of bound variables are called *alpha equivalent*, and are considered "the same." Lean recognizes this equivalence.
+
+Notice that applying a term ``t : α → β`` to a term ``s : α`` yields an expression ``t s : β``.
+Returning to the previous example and renaming bound variables for clarity, notice the types of the following expressions:
+
+```lean
+#check (fun x : Nat => x) 1     -- Nat
+#check (fun x : Nat => true) 1  -- Bool
+
+constant f : Nat → String
+constant g : String → Bool
+
+#check
+  (fun (α β γ : Type) (g : β → γ) (f : α → β) (x : α) => g (f x)) Nat String Bool g f 0
+  -- Bool
+```
+
+As expected, the expression ``(fun x : Nat =>  x) 1`` has type ``Nat``.
+In fact, more should be true: applying the expression ``(fun x : Nat => x)`` to ``1`` should "return" the value ``1``. And, indeed, it does:
+
+```lean
+#reduce (fun x : Nat => x) 1     -- 1
+#reduce (fun x : Nat => true) 1  -- true
+
+constant f : Nat → String
+constant g : String → Bool
+
+#reduce
+  (fun (α β γ : Type) (g : β → γ) (f : α → β) (x : α) => g (f x)) Nat String Bool g f 0
+  -- g (f 0)
+```
+
+The command ``#reduce`` tells Lean to evaluate an expression by *reducing* it to its normal form,
+which is to say, carrying out all the computational reductions that are sanctioned by its kernel.
+The process of simplifying an expression ``(fun x => t) s`` to ``t[s/x]`` -- that is, ``t`` with ``s`` substituted for the variable ``x`` --
+is known as *beta reduction*, and two terms that beta reduce to a common term are called *beta equivalent*.
+But the ``#reduce`` command carries out other forms of reduction as well:
+
+```lean
+constant m : Nat
+constant n : Nat
+constant b : Bool
+
+#reduce (m, n).1        -- m
+#reduce (m, n).2        -- n
+
+#reduce true && false   -- false
+#reduce false && b      -- false
+#reduce b && false      -- Bool.rec false false b
+
+#reduce n + 0           -- n
+#reduce n + 2           -- Nat.succ (Nat.succ n)
+#reduce 2 + 3           -- 5
+```
+
+We explain later how these terms are evaluated.
+For now, we only wish to emphasize that this is an important feature of dependent type theory:
+every term has a computational behavior, and supports a notion of reduction, or *normalization*.
+In principle, two terms that reduce to the same value are called *definitionally equal*.
+They are considered "the same" by Lean's type checker, and Lean does its best to recognize and support these identifications.
+The `#reduce` command is mainly useful to understand why two terms are considered the same.
+
+Lean is also a programming language. It has a compiler to native code and an interpreter.
+You can use the command `#eval` to execute expressions, and it is the preferred way of testing your functions.
+Note that `#eval` and `#reduce` are *not* equivalent. The command `#eval` first compiles Lean expressions
+into an intermediate representation (IR) and then uses an interpreter to execute the generated IR.
+Some builtin types (e.g., `Nat`, `String`, `Array`) have a more efficient representation in the IR.
+The IR has support for using foreign functions that are opaque to Lean.
+
+In contrast, the ``#reduce`` command relies on a reduction engine similar to the one used in Lean's trusted kernel,
+the part of Lean that is responsible for checking and verifying the correctness of expressions and proofs.
+It is less efficient than ``#eval``, and treats all foreign functions as opaque constants.
+We later discuss other differences between the two commands.

doc/functions.md

@@ -0,0 +1,153 @@
+# Functions
+
+Functions are the fundamental unit of program execution in any programming language.
+As in other languages, a Lean function has a name, can have parameters and take arguments, and has a body.
+Lean also supports functional programming constructs such as treating functions as values,
+using unnamed functions in expressions, composition of functions to form new functions,
+curried functions, and the implicit definition of functions by way of
+the partial application of function arguments.
+
+You define functions by using the `def` keyword followed by its name, a parameter list, return type and its body.
+The parameter list consists of successive parameters that are separated by spaces.
+You can specify an explicit type for each parameter.
+If you do not specify a specific argument type, the compiler tries to infer the type from the function body.
+An error is returned when it cannot be inferred.
+The expression that makes up the function body is typically a compound expression consisting of a number of expressions
+that culminate in a final expression that is the return value.
+The return type is a colon followed by a type and is optional.
+If you do not specify the type of the return value explicitly,
+the compiler tries to determine the return type from the final expression.
+
+```lean
+def f x := x + 1
+```
+In the previous example, the function name is `f`, the argument is `x`, which has type `Nat`,
+the function body is `x + 1`, and the return value is of type `Nat`.
+The following example defines the factorial recursive function using pattern matching.
+```lean
+def fact x :=
+  match x with
+  | 0   => 1
+  | n+1 => (n+1) * fact n
+
+#eval fact 100
+```
+By default, Lean only accepts total functions.
+The `partial` keyword may be used to define a recursive function without a termination proof; `partial` functions compute in compiled programs, but are opaque in proofs and during type checking.
+```lean
+partial def g (x : Nat) (p : Nat -> Bool) : Nat :=
+  if p x then
+    x
+  else
+    g (x+1) p
+
+#eval g 0 (fun x => x > 10)
+```
+In the previous example, `g x p` only terminates if there is a `y >= x` such that `p y` returns `true`.
+Of course, `g 0 (fun x => false)` never terminates.
+
+However, the use of `partial` is restricted to functions whose return type is not empty so the soundness
+of the system is not compromised.
+
+```lean,ignore
+partial def loop? : α := -- failed to compile partial definition 'loop?', failed to
+  loop?                  -- show that type is inhabited and non empty
+
+partial def loop [Inhabited α] : α := -- compiles
+  loop
+
+example : True := -- accepted
+  loop
+
+example : False :=
+  loop -- failed to synthesize instance Inhabited False
+```
+
+If we were able to partially define `loop?`, we could prove `False` with it.
+
+# Lambda expressions
+
+A lambda expression is an unnamed function.
+You define lambda expressions by using the `fun` keyword. A lambda expression resembles a function definition, except that instead of the `:=` token,
+the `=>` token is used to separate the argument list from the function body. As in a regular function definition,
+the argument types can be inferred or specified explicitly, and the return type of the lambda expression is inferred from the type of the
+last expression in the body.
+
+```lean
+def twice (f : Nat -> Nat) (x : Nat) : Nat :=
+  f (f x)
+
+#eval twice (fun x => x + 1) 3
+#eval twice (fun (x : Nat) => x * 2) 3
+
+#eval List.map (fun x => x + 1) [1, 2, 3]
+-- [2, 3, 4]
+
+#eval List.map (fun (x, y) => x + y) [(1, 2), (3, 4)]
+-- [3, 7]
+```
+
+# Syntax sugar for simple lambda expressions
+
+Simple functions can be defined using parentheses and `·` as a placeholder.
+```lean
+#check (· + 1)
+-- fun a => a + 1
+#check (2 - ·)
+-- fun a => 2 - a
+#eval [1, 2, 3, 4, 5].foldl (· * ·) 1
+-- 120
+
+def h (x y z : Nat) :=
+  x + y + z
+
+#check (h · 1 ·)
+-- fun a b => h a 1 b
+
+#eval [(1, 2), (3, 4), (5, 6)].map (·.1)
+-- [1, 3, 5]
+```
+In the previous example, the term `(·.1)` is syntax sugar for `fun x => x.1`.
+
+# Pipelining
+
+Pipelining enables function calls to be chained together as successive operations. Pipelining works as follows:
+
+```lean
+def add1 x := x + 1
+def times2 x := x * 2
+
+#eval times2 (add1 100)
+#eval 100 |> add1 |> times2
+#eval times2 <| add1 <| 100
+```
+The result of the previous `#eval` commands is 202.
+The forward pipeline `|>` operator takes a function and an argument and return a value.
+In contrast, the backward pipeline `<|` operator takes an argument and a function and returns a value.
+These operators are useful for minimizing the number of parentheses.
+```lean
+def add1Times3FilterEven (xs : List Nat) :=
+  List.filter (· % 2 == 0) (List.map (· * 3) (List.map (· + 1) xs))
+
+#eval add1Times3FilterEven [1, 2, 3, 4]
+-- [6, 12]
+
+-- Define the same function using pipes
+def add1Times3FilterEven' (xs : List Nat) :=
+  xs |> List.map (· + 1) |> List.map (· * 3) |> List.filter (· % 2 == 0)
+
+#eval add1Times3FilterEven' [1, 2, 3, 4]
+-- [6, 12]
+```
+Lean also supports the operator `|>.` which combines forward pipeline `|>` operator with the `.` field notation.
+```lean
+-- Define the same function using pipes
+def add1Times3FilterEven'' (xs : List Nat) :=
+  xs.map (· + 1) |>.map (· * 3) |>.filter (· % 2 == 0)
+
+#eval add1Times3FilterEven'' [1, 2, 3, 4]
+-- [6, 12]
+```
+
+For users familiar with the Haskell programming language,
+Lean also supports the notation `f $ a` for the backward pipeline `f <| a`.

doc/implicit.md

@@ -0,0 +1,142 @@
+## Implicit Arguments
+
+Suppose we define the `compose` function as.
+
+```lean
+def compose (α β γ : Type) (g : β → γ) (f : α → β) (x : α) : γ :=
+  g (f x)
+```
+
+The function `compose` takes three types, ``α``, ``β``, and ``γ``, and two functions, ``g : β → γ`` and ``f : α → β``, a value `x : α`, and
+returns ``g (f x)``, the composition of ``g`` and ``f``.
+We say `compose` is polymorphic over types ``α``, ``β``, and ``γ``. Now, let's use `compose`:
+
+```lean
+# def compose (α β γ : Type) (g : β → γ) (f : α → β) (x : α) : γ :=
+#   g (f x)
+def double (x : Nat) := 2*x
+def triple (x : Nat) := 3*x
+
+#check compose Nat Nat Nat double triple 10 -- Nat
+#eval  compose Nat Nat Nat double triple 10 -- 60
+
+def appendWorld (s : String) := s ++ "world"
+#check String.length -- String → Nat
+
+#check compose String String Nat String.length appendWorld "hello" -- Nat
+#eval  compose String String Nat String.length appendWorld "hello" -- 10
+```
+
+Because `compose` is polymorphic over types ``α``, ``β``, and ``γ``, we have to provide them in the examples above.
+But this information is redundant: one can infer the types from the arguments ``g`` and ``f``.
+This is a central feature of dependent type theory: terms carry a lot of information, and often some of that information can be inferred from the context.
+In Lean, one uses an underscore, ``_``, to specify that the system should fill in the information automatically.
+
+```lean
+# def compose (α β γ : Type) (g : β → γ) (f : α → β) (x : α) : γ :=
+#  g (f x)
+# def double (x : Nat) := 2*x
+# def triple (x : Nat) := 3*x
+#check compose _ _ _ double triple 10 -- Nat
+#eval  compose Nat Nat Nat double triple 10 -- 60
+# def appendWorld (s : String) := s ++ "world"
+# #check String.length -- String → Nat
+#check compose _ _ _ String.length appendWorld "hello" -- Nat
+#eval  compose _ _ _ String.length appendWorld "hello" -- 10
+```
+It is still tedious, however, to type all these underscores. When a function takes an argument that can generally be inferred from context,
+Lean allows us to specify that this argument should, by default, be left implicit. This is done by putting the arguments in curly braces, as follows:
+
+```lean
+def compose {α β γ : Type} (g : β → γ) (f : α → β) (x : α) : γ :=
+  g (f x)
+# def double (x : Nat) := 2*x
+# def triple (x : Nat) := 3*x
+#check compose double triple 10 -- Nat
+#eval  compose double triple 10 -- 60
+# def appendWorld (s : String) := s ++ "world"
+# #check String.length -- String → Nat
+#check compose String.length appendWorld "hello" -- Nat
+#eval  compose String.length appendWorld "hello" -- 10
+```
+All that has changed are the braces around ``α β γ: Type``.
+It makes these three arguments implicit. Notationally, this hides the specification of the type,
+making it look as though ``compose`` simply takes 3 arguments.
+
+Variables can also be specified as implicit when they are declared with
+the ``variable`` command:
+
+```lean
+universe u
+
+section
+  variable {α : Type u}
+  variable (x : α)
+  def ident := x
+end
+
+variable (α β : Type u)
+variable (a : α) (b : β)
+
+#check ident
+#check ident a
+#check ident b
+```
+
+This definition of ``ident`` here has the same effect as the one above.
+
+Lean has very complex mechanisms for instantiating implicit arguments, and we will see that they can be used to infer function types, predicates, and even proofs.
+The process of instantiating these "holes," or "placeholders," in a term is part of a bigger process called *elaboration*.
+The presence of implicit arguments means that at times there may be insufficient information to fix the meaning of an expression precisely.
+An expression like ``ident`` is said to be *polymorphic*, because it can take on different meanings in different contexts.
+
+One can always specify the type ``T`` of an expression ``e`` by writing ``(e : T)``.
+This instructs Lean's elaborator to use the value ``T`` as the type of ``e`` when trying to elaborate it.
+In the following example, this mechanism is used to specify the desired types of the expressions ``ident``.
+
+```lean
+def ident {α : Type u} (a : α) : α := a
+
+#check (ident : Nat → Nat) -- Nat → Nat
+```
+
+Numerals are overloaded in Lean, but when the type of a numeral cannot be inferred, Lean assumes, by default, that it is a natural number.
+So the expressions in the first two ``#check`` commands below are elaborated in the same way, whereas the third ``#check`` command interprets ``2`` as an integer.
+
+```lean
+#check 2         -- Nat
+#check (2 : Nat) -- Nat
+#check (2 : Int) -- Int
+```
+
+Sometimes, however, we may find ourselves in a situation where we have declared an argument to a function to be implicit,
+but now want to provide the argument explicitly. If ``foo`` is such a function, the notation ``@foo`` denotes the same function with all
+the arguments made explicit.
+
+```lean
+# def ident {α : Type u} (a : α) : α := a
+variable (α β : Type)
+
+#check @ident           -- {α : Type u} → α → α
+#check @ident α         -- α → α
+#check @ident β         -- β → β
+#check @ident Nat       -- Nat → Nat
+#check @ident Bool true -- Bool
+```
+
+Notice that now the first ``#check`` command gives the type of the identifier, ``ident``, without inserting any placeholders.
+Moreover, the output indicates that the first argument is implicit.
+
+Named arguments enable you to specify an argument for a parameter by matching the argument with
+its name rather than with its position in the parameter list. You can use them to specify explicit *and* implicit arguments.
+If you don't remember the order of the parameters but know their names, you can send the arguments in any order.
+You may also provide the value for an implicit parameter when
+Lean failed to infer it. Named arguments also improve the readability of your code by identifying what
+each argument represents.
+
+```lean
+# def ident {α : Type u} (a : α) : α := a
+
+#check ident (α := Nat)  -- Nat → Nat
+#check ident (α := Bool) -- Bool → Bool
+```

doc/inductive.md

@@ -0,0 +1,3 @@
+# Inductive Types
+
+[Theorem Proving in Lean](https://lean-lang.org/theorem_proving_in_lean4/inductive_types.html) has a chapter about inductive datatypes.

doc/int.md

@@ -0,0 +1,37 @@
+# Integers
+
+The `Int` type represents the arbitrary-precision integers. There are no overflows.
+```lean
+#eval (100000000000000000 : Int) * 200000000000000000000 * 1000000000000000000000
+```
+Recall that nonnegative numerals are considered to be a `Nat` if there are no typing constraints.
+```lean
+#check 1 -- Nat
+#check -1 -- Int
+#check (1:Int) -- Int
+```
+
+The operator `/` for `Int` implements integer division.
+```lean
+#eval -10 / 4 -- -3
+```
+
+Similar to `Nat`, the internal representation of `Int` is optimized. Small integers are
+represented by a single machine word. Big integers are implemented using [GMP](https://gmplib.org/manual/) numbers.
+We recommend you use fixed precision numeric types only in performance critical code.
+
+The Lean kernel does not have special support for reducing `Int` during type checking.
+However, since `Int` is defined as
+```lean
+# namespace hidden
+inductive Int : Type where
+  | ofNat   : Nat → Int
+  | negSucc : Nat → Int
+# end hidden
+```
+the type checker will be able reduce `Int` expressions efficiently by relying on the special support for `Nat`.
+
+```lean
+theorem ex : -2000000000 * 1000000000 = -2000000000000000000 :=
+ rfl
+```

doc/introdef.md

@@ -0,0 +1,66 @@
+## Introducing Definitions
+
+The ``def`` command provides one important way of defining new objects.
+
+```lean
+
+def foo : (Nat → Nat) → Nat :=
+  fun f => f 0
+
+#check foo   -- (Nat → Nat) → Nat
+#print foo
+```
+
+We can omit the type when Lean has enough information to infer it:
+
+```lean
+def foo :=
+  fun (f : Nat → Nat) => f 0
+```
+
+The general form of a definition is ``def foo : α := bar``. Lean can usually infer the type ``α``, but it is often a good idea to write it explicitly.
+This clarifies your intention, and Lean will flag an error if the right-hand side of the definition does not have the right type.
+
+Lean also allows us to use an alternative format that puts the abstracted variables before the colon and omits the lambda:
+```lean
+def double (x : Nat) : Nat :=
+  x + x
+
+#print double
+#check double 3
+#reduce double 3  -- 6
+#eval double 3    -- 6
+
+def square (x : Nat) :=
+  x * x
+
+#print square
+#check square 3
+#reduce square 3  -- 9
+#eval square 3    -- 9
+
+def doTwice (f : Nat → Nat) (x : Nat) : Nat :=
+  f (f x)
+
+#eval doTwice double 2   -- 8
+```
+
+These definitions are equivalent to the following:
+
+```lean
+def double : Nat → Nat :=
+  fun x => x + x
+
+def square : Nat → Nat :=
+  fun x => x * x
+
+def doTwice : (Nat → Nat) → Nat → Nat :=
+  fun f x => f (f x)
+```
+
+We can even use this approach to specify arguments that are types:
+
+```lean
+def compose (α β γ : Type) (g : β → γ) (f : α → β) (x : α) : γ :=
+  g (f x)
+```

doc/lean3changes.md

@@ -0,0 +1,369 @@
+# Significant changes from Lean 3
+
+Lean 4 is not backward compatible with Lean 3.
+We have rewritten most of the system, and took the opportunity to cleanup the syntax,
+metaprogramming framework, and elaborator. In this section, we go over the most significant
+changes.
+
+## Lambda expressions
+
+We do not use `,` anymore to separate the binders from the lambda expression body.
+The Lean 3 syntax for lambda expressions was unconventional, and `,` has been overused in Lean 3.
+For example, we believe a list of lambda expressions is quite confusing in Lean 3, since `,` is used
+to separate the elements of a list, and in the lambda expression itself. We now use `=>` as the separator,
+as an example, `fun x => x` is the identity function. One may still use the symbol `λ` as a shorthand for `fun`.
+The lambda expression notation has many new features that are not supported in Lean 3.
+
+## Pattern matching
+
+In Lean 4, one can easily create new notation that abbreviates commonly used idioms. One of them is a
+`fun` followed by a `match`. In the following examples, we define a few functions using `fun`+`match` notation.
+
+```lean
+# namespace ex1
+def Prod.str : Nat × Nat → String :=
+  fun (a, b) => "(" ++ toString a ++ ", " ++ toString b ++ ")"
+
+structure Point where
+  x : Nat
+  y : Nat
+  z : Nat
+
+def Point.addX : Point → Point → Nat :=
+  fun { x := a, .. } { x := b, .. } =>  a+b
+
+def Sum.str : Option Nat → String :=
+  fun
+    | some a => "some " ++ toString a
+    | none   => "none"
+# end ex1
+```
+
+## Implicit lambdas
+
+In Lean 3 stdlib, we find many [instances](https://github.com/leanprover/lean/blob/master/library/init/category/reader.lean#L39) of the dreadful `@`+`_` idiom.
+It is often used when the expected type is a function type with implicit arguments,
+and we have a constant (`reader_t.pure` in the example) which also takes implicit arguments. In Lean 4, the elaborator automatically introduces lambdas
+for consuming implicit arguments. We are still exploring this feature and analyzing its impact, but the experience so far has been very positive. As an example,
+here is the example in the link above using Lean 4 implicit lambdas.
+
+```lean
+# variable (ρ : Type) (m : Type → Type) [Monad m]
+instance : Monad (ReaderT ρ m) where
+  pure := ReaderT.pure
+  bind := ReaderT.bind
+```
+
+Users can disable the implicit lambda feature by using `@` or writing a lambda expression with `{}` or `[]` binder annotations.
+Here are few examples
+
+```lean
+# namespace ex2
+def id1 : {α : Type} → α → α :=
+  fun x => x
+
+def listId : List ({α : Type} → α → α) :=
+  (fun x => x) :: []
+
+-- In this example, implicit lambda introduction has been disabled because
+-- we use `@` before `fun`
+def id2 : {α : Type} → α → α :=
+  @fun α (x : α) => id1 x
+
+def id3 : {α : Type} → α → α :=
+  @fun α x => id1 x
+
+def id4 : {α : Type} → α → α :=
+  fun x => id1 x
+
+-- In this example, implicit lambda introduction has been disabled
+-- because we used the binder annotation `{...}`
+def id5 : {α : Type} → α → α :=
+  fun {α} x => id1 x
+# end ex2
+```
+
+## Sugar for simple functions
+
+In Lean 3, we can create simple functions from infix operators by using parentheses. For example, `(+1)` is sugar for `fun x, x + 1`. In Lean 4, we generalize this notation using `·` as a placeholder. Here are a few examples:
+
+```lean
+# namespace ex3
+#check (· + 1)
+-- fun a => a + 1
+#check (2 - ·)
+-- fun a => 2 - a
+#eval [1, 2, 3, 4, 5].foldl (·*·) 1
+-- 120
+
+def f (x y z : Nat) :=
+  x + y + z
+
+#check (f · 1 ·)
+-- fun a b => f a 1 b
+
+#eval [(1, 2), (3, 4), (5, 6)].map (·.1)
+-- [1, 3, 5]
+# end ex3
+```
+
+As in Lean 3, the notation is activated using parentheses, and the lambda abstraction is created by collecting the nested `·`s.
+The collection is interrupted by nested parentheses. In the following example, two different lambda expressions are created.
+
+```lean
+#check (Prod.mk · (· + 1))
+-- fun a => (a, fun b => b + 1)
+```
+
+## Function applications
+
+In Lean 4, we have support for named arguments.
+Named arguments enable you to specify an argument for a parameter by matching the argument with
+its name rather than with its position in the parameter list.
+If you don't remember the order of the parameters but know their names,
+you can send the arguments in any order. You may also provide the value for an implicit parameter when
+Lean failed to infer it. Named arguments also improve the readability of your code by identifying what
+each argument represents.
+
+```lean
+def sum (xs : List Nat) :=
+  xs.foldl (init := 0) (·+·)
+
+#eval sum [1, 2, 3, 4]
+-- 10
+
+example {a b : Nat} {p : Nat → Nat → Nat → Prop} (h₁ : p a b b) (h₂ : b = a)
+    : p a a b :=
+  Eq.subst (motive := fun x => p a x b) h₂ h₁
+```
+In the following examples, we illustrate the interaction between named and default arguments.
+
+```lean
+def f (x : Nat) (y : Nat := 1) (w : Nat := 2) (z : Nat) :=
+  x + y + w - z
+
+example (x z : Nat) : f (z := z) x = x + 1 + 2 - z := rfl
+
+example (x z : Nat) : f x (z := z) = x + 1 + 2 - z := rfl
+
+example (x y : Nat) : f x y = fun z => x + y + 2 - z := rfl
+
+example : f = (fun x z => x + 1 + 2 - z) := rfl
+
+example (x : Nat) : f x = fun z => x + 1 + 2 - z := rfl
+
+example (y : Nat) : f (y := 5) = fun x z => x + 5 + 2 - z := rfl
+
+def g {α} [Add α] (a : α) (b? : Option α := none) (c : α) : α :=
+  match b? with
+  | none   => a + c
+  | some b => a + b + c
+
+variable {α} [Add α]
+
+example : g = fun (a c : α) => a + c := rfl
+
+example (x : α) : g (c := x) = fun (a : α) => a + x := rfl
+
+example (x : α) : g (b? := some x) = fun (a c : α) => a + x + c := rfl
+
+example (x : α) : g x = fun (c : α) => x + c := rfl
+
+example (x y : α) : g x y = fun (c : α) => x + y + c := rfl
+```
+
+In Lean 4, we can use `..` to provide missing explicit arguments as `_`.
+This feature combined with named arguments is useful for writing patterns. Here is an example:
+```lean
+inductive Term where
+  | var    (name : String)
+  | num    (val : Nat)
+  | add    (fn : Term) (arg : Term)
+  | lambda (name : String) (type : Term) (body : Term)
+
+def getBinderName : Term → Option String
+  | Term.lambda (name := n) .. => some n
+  | _ => none
+
+def getBinderType : Term → Option Term
+  | Term.lambda (type := t) .. => some t
+  | _ => none
+```
+Ellipsis are also useful when explicit argument can be automatically inferred by Lean, and we want
+to avoid a sequence of `_`s.
+```lean
+example (f : Nat → Nat) (a b c : Nat) : f (a + b + c) = f (a + (b + c)) :=
+  congrArg f (Nat.add_assoc ..)
+```
+
+In Lean 4, writing `f(x)` in place of `f x` is no longer allowed, you must use whitespace between the function and its arguments (e.g., `f (x)`).
+
+## Dependent function types
+
+Given `α : Type` and `β : α → Type`, `(x : α) → β x` denotes the type of functions `f` with the property that,
+for each `a : α`, `f a` is an element of `β a`. In other words, the type of the value returned by `f` depends on its input.
+We say `(x : α) → β x` is a dependent function type. In Lean 3, we write the dependent function type `(x : α) → β x` using
+one of the following three equivalent notations:
+`forall x : α, β x` or `∀ x : α, β x` or `Π x : α, β x`.
+The first two were intended to be used for writing propositions, and the latter for writing code.
+Although the notation `Π x : α, β x` has historical significance, we have removed it from Lean 4 because
+it is awkward to use and often confuses new users. We can still write `forall x : α, β x` and `∀ x : α, β x`.
+
+```lean
+#check forall (α : Type), α → α
+#check ∀ (α : Type), α → α
+#check ∀ α : Type, α → α
+#check ∀ α, α → α
+#check (α : Type) → α → α
+#check {α : Type} → (a : Array α) → (i : Nat) → i < a.size → α
+#check {α : Type} → [ToString α] → α → String
+#check forall {α : Type} (a : Array α) (i : Nat), i < a.size → α
+#check {α β : Type} → α → β → α × β
+```
+
+## The `meta` keyword
+
+In Lean 3, the keyword `meta` is used to mark definitions that can use primitives implemented in C/C++.
+These metadefinitions can also call themselves recursively, relaxing the termination
+restriction imposed by ordinary type theory. Metadefinitions may also use unsafe primitives such as
+`eval_expr (α : Type u) [reflected α] : expr → tactic α`, or primitives that break referential transparency
+`tactic.unsafe_run_io`.
+
+The keyword `meta` has been currently removed from Lean 4. However, we may re-introduce it in the future,
+but with a much more limited purpose: marking meta code that should not be included in the executables produced by Lean.
+
+The keyword `constant` has been deleted in Lean 4, and `axiom` should be used instead. In Lean 4, the new command `opaque` is used to define an opaque definition. Here are two simple examples:
+```lean
+# namespace meta1
+opaque x : Nat := 1
+-- The following example will not type check since `x` is opaque
+-- example : x = 1 := rfl
+
+-- We can evaluate `x`
+#eval x
+-- 1
+
+-- When no value is provided, the elaborator tries to build one automatically for us
+-- using the `Inhabited` type class
+opaque y : Nat
+# end meta1
+```
+
+We can instruct Lean to use a foreign function as the implementation for any definition
+using the attribute `@[extern "foreign_function"]`. It is the user's responsibility to ensure the
+foreign implementation is correct.
+However, a user mistake here will only impact the code generated by Lean, and
+it will **not** compromise the logical soundness of the system.
+That is, you cannot prove `False` using the `@[extern]` attribute.
+We use `@[extern]` with definitions when we want to provide a reference implementation in Lean
+that can be used for reasoning. When we write a definition such as
+```lean
+@[extern "lean_nat_add"]
+def add : Nat → Nat → Nat
+  | a, Nat.zero   => a
+  | a, Nat.succ b => Nat.succ (add a b)
+```
+Lean assumes that the foreign function `lean_nat_add` implements the reference implementation above.
+
+The `unsafe` keyword allows us to define functions using unsafe features such as general recursion,
+and arbitrary type casting. Regular (safe) functions cannot directly use `unsafe` ones since it would
+compromise the logical soundness of the system. As in regular programming languages, programs written
+using unsafe features may crash at runtime. Here are a few unsafe examples:
+```lean
+unsafe def unsound : False :=
+  unsound
+
+#check @unsafeCast
+-- {α : Type _} → {β : Type _} → α → β
+
+unsafe def nat2String (x : Nat) : String :=
+  unsafeCast x
+
+-- The following definition doesn't type check because it is not marked as `unsafe`
+-- def nat2StringSafe (x : Nat) : String :=
+--   unsafeCast x
+```
+
+The `unsafe` keyword is particularly useful when we want to take advantage of an implementation detail of the
+Lean execution runtime. For example, we cannot prove in Lean that arrays have a maximum size, but
+the runtime used to execute Lean programs guarantees that an array cannot have more than 2^64 (2^32) elements
+in a 64-bit (32-bit) machine. We can take advantage of this fact to provide a more efficient implementation for
+array functions. However, the efficient version would not be very useful if it can only be used in
+unsafe code. Thus, Lean 4 provides the attribute `@[implemented_by functionName]`. The idea is to provide
+an unsafe (and potentially more efficient) version of a safe definition or constant. The function `f`
+at the attribute `@[implemented_by f]` is very similar to an extern/foreign function,
+the key difference is that it is implemented in Lean itself. Again, the logical soundness of the system
+cannot be compromised by using the attribute `implemented_by`, but if the implementation is incorrect your
+program may crash at runtime. In the following example, we define `withPtrUnsafe a k h` which
+executes `k` using the memory address where `a` is stored in memory. The argument `h` is proof
+that `k` is a constant function. Then, we "seal" this unsafe implementation at `withPtr`. The proof `h`
+ensures the reference implementation `k 0` is correct. For more information, see the article
+"Sealing Pointer-Based Optimizations Behind Pure Functions".
+```lean
+unsafe
+def withPtrUnsafe {α β : Type} (a : α) (k : USize → β) (h : ∀ u, k u = k 0) : β :=
+  k (ptrAddrUnsafe a)
+
+@[implemented_by withPtrUnsafe]
+def withPtr {α β : Type} (a : α) (k : USize → β) (h : ∀ u, k u = k 0) : β :=
+  k 0
+```
+
+General recursion is very useful in practice, and it would be impossible to implement Lean 4 without it.
+The keyword `partial` implements a very simple and efficient approach for supporting general recursion.
+Simplicity was key here because of the bootstrapping problem. That is, we had to implement Lean in Lean before
+many of its features were implemented (e.g., the tactic framework or support for wellfounded recursion).
+Another requirement for us was performance. Functions tagged with `partial` should be as efficient as the ones implemented in mainstream functional programming
+languages such as OCaml. When the `partial` keyword is used, Lean generates an auxiliary `unsafe` definition that
+uses general recursion, and then defines an opaque constant that is implemented by this auxiliary definition.
+This is very simple, efficient, and is sufficient for users that want to use Lean as a regular programming language.
+A `partial` definition cannot use unsafe features such as `unsafeCast` and `ptrAddrUnsafe`, and it can only be used to
+implement types we already known to be inhabited. Finally, since we "seal" the auxiliary definition using an opaque
+constant, we cannot reason about `partial` definitions.
+
+We are aware that proof assistants such as Isabelle provide a framework for defining partial functions that does not
+prevent users from proving properties about them. This kind of framework can be implemented in Lean 4. Actually,
+it can be implemented by users since Lean 4 is an extensible system. The developers current have no plans to implement
+this kind of support for Lean 4. However, we remark that users can implement it using a function that traverses
+the auxiliary unsafe definition generated by Lean, and produces a safe one using an approach similar to the one used in Isabelle.
+
+```lean
+# namespace partial1
+partial def f (x : Nat) : IO Unit := do
+  IO.println x
+  if x < 100 then
+     f (x+1)
+
+#eval f 98
+# end partial1
+```
+
+## Library changes
+
+These are changes to the library which may trip up Lean 3 users:
+
+- `List` is no longer a monad.
+
+## Style changes
+
+Coding style changes have also been made:
+
+- Term constants and variables are now `lowerCamelCase` rather than `snake_case`
+- Type constants are now `UpperCamelCase`, eg `Nat`, `List`. Type variables are still lower case greek letters. Functors are still lower case latin `(m : Type → Type) [Monad m]`.
+- When defining typeclasses, prefer not to use "has". Eg `ToString` or `Add` instead of `HasToString` or `HasAdd`.
+- Prefer `return` to `pure` in monad expressions.
+- Pipes `<|` are preferred to dollars `$` for function application.
+- Declaration bodies should always be indented:
+  ```lean
+  inductive Hello where
+    | foo
+    | bar
+
+  structure Point where
+    x : Nat
+    y : Nat
+
+  def Point.addX : Point → Point → Nat :=
+    fun { x := a, .. } { x := b, .. } => a + b
+  ```
+- In structures and typeclass definitions, prefer `where` to `:=` and don't surround fields with parentheses. (Shown in `Point` above)

doc/lexical_structure.md

@@ -0,0 +1,179 @@
+Lexical Structure
+=================
+
+This section describes the detailed lexical structure of the Lean
+language.
+
+A Lean program consists of a stream of UTF-8 tokens where each token
+is one of the following:
+
+```
+   token: symbol | command | ident | string | raw_string | char | numeral |
+        : decimal | doc_comment | mod_doc_comment | field_notation
+```
+
+Tokens can be separated by the whitespace characters space, tab, line
+feed, and carriage return, as well as comments. Single-line comments
+start with ``--``, whereas multi-line comments are enclosed by ``/-``
+and ``-/`` and can be nested.
+
+Symbols and Commands
+====================
+
+.. *(TODO: list built-in symbols and command tokens?)*
+
+Symbols are static tokens that are used in term notations and
+commands. They can be both keyword-like (e.g. the `have
+<structured_proofs>` keyword) or use arbitrary Unicode characters.
+
+Command tokens are static tokens that prefix any top-level declaration
+or action. They are usually keyword-like, with transitory commands
+like `#print <instructions>` prefixed by the ``#`` character. The set
+of built-in commands is listed in [Other Commands](./other_commands.md).
+
+Users can dynamically extend the sets of both symbols (via the
+commands listed in [Quoted Symbols](#quoted-symbols) and command
+tokens (via the `[user_command] <attributes>` attribute).
+
+.. _identifiers:
+
+Identifiers
+===========
+
+An *atomic identifier*, or *atomic name*, is (roughly) an alphanumeric
+string that does not begin with a numeral. A (hierarchical)
+*identifier*, or *name*, consists of one or more atomic names
+separated by periods.
+
+Parts of atomic names can be escaped by enclosing them in pairs of French double quotes ``«»``.
+
+```lean
+   def Foo.«bar.baz» := 0  -- name parts ["Foo", "bar.baz"]
+```
+
+```
+   ident: atomic_ident | ident "." atomic_ident
+   atomic_ident: atomic_ident_start atomic_ident_rest*
+   atomic_ident_start: letterlike | "_" | escaped_ident_part
+   letterlike: [a-zA-Z] | greek | coptic | letterlike_symbols
+   greek: <[α-ωΑ-Ωἀ-῾] except for [λΠΣ]>
+   coptic: [ϊ-ϻ]
+   letterlike_symbols: [℀-⅏]
+   escaped_ident_part: "«" [^«»\r\n\t]* "»"
+   atomic_ident_rest: atomic_ident_start | [0-9'ⁿ] | subscript
+   subscript: [₀-₉ₐ-ₜᵢ-ᵪ]
+```
+
+String Literals
+===============
+
+String literals are enclosed by double quotes (``"``). They may contain line breaks, which are conserved in the string value.  Backslash (`\`) is a special escape character which can be used to the following
+special characters:
+- `\\` represents an escaped backslash, so this escape causes one backslash to be included in the string.
+- `\"` puts a double quote in the string.
+- `\'` puts an apostrophe in the string.
+- `\n` puts a new line character in the string.
+- `\t` puts a tab character in the string.
+- `\xHH` puts the character represented by the 2 digit hexadecimal into the string.  For example
+"this \x26 that" which become "this & that".  Values above 0x80 will be interpreted according to the
+[Unicode table](https://unicode-table.com/en/) so "\xA9 Copyright 2021" is "© Copyright 2021".
+- `\uHHHH` puts the character represented by the 4 digit hexadecimal into the string, so the following
+string "\u65e5\u672c" will become "日本" which means "Japan".
+- `\` followed by a newline and then any amount of whitespace is a "gap" that is equivalent to the empty string,
+useful for letting a string literal span across multiple lines. Gaps spanning multiple lines can be confusing,
+so the parser raises an error if the trailing whitespace contains any newlines.
+
+So the complete syntax is:
+
+```
+   string       : '"' string_item '"'
+   string_item  : string_char | char_escape | string_gap
+   string_char  : [^"\\]
+   char_escape  : "\" ("\" | '"' | "'" | "n" | "t" | "x" hex_char{2} | "u" hex_char{4})
+   hex_char     : [0-9a-fA-F]
+   string_gap   : "\" newline whitespace*
+```
+
+Raw String Literals
+===================
+
+Raw string literals are string literals without any escape character processing.
+They begin with `r##...#"` (with zero or more `#` characters) and end with `"#...##` (with the same number of `#` characters).
+The contents of a raw string literal may contain `"##..#` so long as the number of `#` characters
+is less than the number of `#` characters used to begin the raw string literal.
+
+```
+   raw_string          : raw_string_aux(0) | raw_string_aux(1) | raw_string_aux(2) | ...
+   raw_string_aux(n)   : 'r' '#'{n} '"' raw_string_item '"' '#'{n}
+   raw_string_item(n)  : raw_string_char | raw_string_quote(n)
+   raw_string_char     : [^"]
+   raw_string_quote(n) : '"' '#'{0..n-1}
+```
+
+Char Literals
+=============
+
+Char literals are enclosed by single quotes (``'``).
+
+```
+   char      : "'" char_item "'"
+   char_item : char_char | char_escape
+   char_char : [^'\\]
+```
+
+Numeric Literals
+================
+
+Numeric literals can be specified in various bases.
+
+```
+   numeral    : numeral10 | numeral2 | numeral8 | numeral16
+   numeral10  : [0-9]+
+   numeral2   : "0" [bB] [0-1]+
+   numeral8   : "0" [oO] [0-7]+
+   numeral16  : "0" [xX] hex_char+
+```
+
+Floating point literals are also possible with optional exponent:
+
+```
+   float    : [0-9]+ "." [0-9]+ [[eE[+-][0-9]+]
+```
+
+For example:
+
+```
+constant w : Int := 55
+constant x : Nat := 26085
+constant y : Nat := 0x65E5
+constant z : Float := 2.548123e-05
+```
+
+Note: that negative numbers are created by applying the "-" negation prefix operator to the number, for example:
+
+```
+constant w : Int := -55
+```
+
+Doc Comments
+============
+
+A special form of comments, doc comments are used to document modules
+and declarations.
+
+```
+   doc_comment: "/--" ([^-] | "-" [^/])* "-/"
+   mod_doc_comment: "/-!" ([^-] | "-" [^/])* "-/"
+```
+
+Field Notation
+==============
+
+Trailing field notation tokens are used in expressions such as
+``(1+1).to_string``. Note that ``a.toString`` is a single
+[Identifier](#identifiers), but may be interpreted as a field
+notation expression by the parser.
+
+```
+   field_notation: "." ([0-9]+ | atomic_ident)
+```

doc/list.md

@@ -0,0 +1 @@
+# List

doc/macro_overview.md

@@ -0,0 +1,393 @@
+
+# Macro Overview
+
+The official paper describing the mechanics behind Lean 4's macro system can be
+found in [Beyond Notations: Hygienic Macro Expansion for Theorem Proving
+Languages](https://arxiv.org/abs/2001.10490) by Sebastian Ullrich and Leonardo
+de Moura, and the accompanying repo with example code can be found in the
+paper's code [supplement](https://github.com/Kha/macro-supplement). The
+supplement also includes a working implementation of the macro expander, so it's
+a good case study for people interested in the details.
+
+## What is a macro in Lean?
+
+A macro is a function that takes in a syntax tree and produces a new syntax
+tree. Macros are useful for many reasons, but two of the big ones are a)
+allowing users to extend the language with new syntactic constructs without
+having to actually expand the core language, and b) allowing users to automate
+tasks that would otherwise be extremely repetitive, time-consuming, and/or
+error-prone.
+
+A motivating example is set builder notation. We would like to be able to write
+the set of natural numbers 0, 1, and 2 as just `{0, 1, 2}`. However, Lean does
+not natively support this syntax, and the actual definition of a set in Mathlib
+does not let us just declare sets in this manner; naively using the set API
+would force us to write `Set.insert 1 (Set.insert 2 (Set.singleton 3))`.
+Instead, we can teach Lean's macro system to recognize `{0, 1, 2}` as a
+shorthand for a composition of existing methods and let it do the repetitive
+work of creating the `Set.insert...` invocation for us. In this way, we can have
+our more readable and more convenient syntax without having to extend Lean
+itself, and while retaining the simple insert/singleton API.
+
+## How macros are handled
+
+The general procedure is as follows:
+
+1. Lean parses a command, creating a Lean syntax tree which contains any
+   unexpanded macros.
+
+2. Lean repeats the cycle (elaboration ~> (macro hygiene and expansion) ~>
+   elaboration...)
+
+The cycle in step 2 repeats until there are no more macros which need to be
+expanded, and elaboration can finish normally. This repetition is required since
+macros can expand to other macros, and may expand to code that needs information
+from the elaborator. As you can see, the process of macro parsing and expansion
+is interleaved with the parsing and elaboration of non-macro code.
+
+By default, macros in Lean are hygienic, which means the system avoids
+accidental name capture when reusing the same name inside and outside the macro.
+Users may occasionally want to disable hygiene, which can be accomplished with
+the command `set_option hygiene false`. More in-depth information about hygiene
+and how it's implemented in the official paper and supplement linked at the top
+of this guide.
+
+## Elements of "a" macro (important types)
+
+
+In the big picture, a macro has two components that must be implemented by the
+user, parsers and syntax transformers, where the latter is a function that says
+what the input syntax should expand to. There is a third component, syntax
+categories, such as `term`, `tactic`, and `command`, but declaring a new syntax
+category is not always necessary. When we say "parser" in the context of a
+macro, we refer to the core type `Lean.ParserDescr`, which parses elements of
+type `Lean.Syntax`, where `Lean.Syntax` represents elements of a Lean syntax
+tree. Syntax transformers are functions of type `Syntax -> MacroM Syntax`. Lean
+has a synonym for this type, which is simply `Macro`. `MacroM` is a monad that
+carries state needed for macro expansion to work nicely, including the info
+needed to implement hygiene.
+
+As an example, we again refer to Mathlib's set builder notation:
+```lean
+/- Declares a parser -/
+syntax (priority := high) "{" term,+ "}" : term
+
+/- Declares two expansions/syntax transformers -/
+macro_rules
+  | `({$x}) => `(Set.singleton $x)
+  | `({$x, $xs:term,*}) => `(Set.insert $x {$xs,*})
+
+/- Provided `Set` has been imported (from Mathlib4), these are all we need for `{1, 2, 3}` to be valid notation to create a literal set -/
+
+```
+
+This example should also make clear the reason why macros (and pretty much all
+of Lean 4's metaprogramming facilities) are functions that take an argument of
+type `Syntax` e.g. `Syntax -> MacroM Syntax`; the leading syntax element is the
+thing that actually triggers the macro expansion by matching with the declared
+parser, and as a user, you will almost always be interested in inspecting and
+transforming that initial syntax element (though there are cases in which it can
+just be ignored, as in the parameter-less exfalso tactic).
+
+Returning briefly to the API provided by Lean, `Lean.Syntax`, is pretty much
+what you would expect a basic syntax tree type to look like. Below is a slightly
+simplified representation which omits details in the `atom` and `ident`
+constructors; users can create atoms and idents which comport with this
+simplified representation using the `mkAtom` and `mkIdent` methods provided in
+the `Lean` namespace.
+```lean
+# open Lean
+inductive Syntax where
+  | missing : Syntax
+  | node (kind : SyntaxNodeKind) (args : Array Syntax) : Syntax
+  | atom : String -> Syntax
+  | ident : Name -> Syntax
+```
+
+
+
+For those interested, `MacroM` is a `ReaderT`:
+```lean
+# open Lean
+abbrev MacroM := ReaderT Macro.Context (EStateM Macro.Exception Macro.State)
+```
+
+The other relevant components are defined as follows:
+```lean
+# open Lean
+structure Context where
+  methods        : MethodsRef
+  mainModule     : Name
+  currMacroScope : MacroScope
+  currRecDepth   : Nat := 0
+  maxRecDepth    : Nat := defaultMaxRecDepth
+  ref            : Syntax
+
+inductive Exception where
+  | error             : Syntax → String → Exception
+  | unsupportedSyntax : Exception
+
+structure State where
+  macroScope : MacroScope
+  traceMsgs  : List (Prod Name String) := List.nil
+  deriving Inhabited
+```
+
+As a review/checklist, the three (sometimes only two depending on whether you
+need a new syntax category) components users need to be concerned with are:
+
+0. You may or may not need to declare a new syntax category using
+   `declare_syntax_cat`
+1. Declare a parser with either `syntax` or `macro`
+2. Declare an expansion/syntax transformer with either `macro_rules` or `macro`
+
+Parsers and syntax transformers can be declared manually, but use of the pattern
+language and `syntax`, `macro_rules`, and `macro` is recommended.
+
+## syntax categories with declare_syntax_cat
+
+`declare_syntax_cat` declares a new syntax category, like `command`, `tactic`,
+or mathlib4's `binderterm`. These are the different categories of things that
+can be referred to in a quote/antiquote. `declare_syntax_cat` results in a call
+to `registerParserCategory` and produces a new parser descriptor:
+
+```lean
+set_option trace.Elab.definition true in
+declare_syntax_cat binderterm
+
+/-
+Output:
+
+[Elab.definition.body] binderterm.quot : Lean.ParserDescr :=
+Lean.ParserDescr.node `Lean.Parser.Term.quot 1024
+  (Lean.ParserDescr.binary `andthen (Lean.ParserDescr.symbol "`(binderterm|")
+    (Lean.ParserDescr.binary `andthen (Lean.ParserDescr.cat `binderterm 0)
+      (Lean.ParserDescr.symbol ")")))
+-/
+```
+
+Declaring a new syntax category like this one automatically declares a quotation
+operator `` `(binderterm| ...)``. These pipe prefixes `<thing>|` are used in
+syntax quotations to say what category a given quotation is expected to be an
+element of. The pipe prefixes are *not* used for elements in the `term` and
+`command` categories (since they're considered the default), but need to be used
+for everything else.
+
+## Parsers and the `syntax` keyword
+
+Internally, elements of type `Lean.ParserDescr` are implemented as parser
+combinators. However, Lean offers the ability to write parsers using the
+macro/pattern language by way of the `syntax` keyword. This is the recommended
+means of writing parsers. As an example, the parser for the `rwa` (rewrite, then
+use assumption) tactic is:
+
+```lean
+# open Lean.Parser.Tactic
+set_option trace.Elab.definition true in
+syntax "rwa " rwRuleSeq (location)? : tactic
+
+/-
+which expands to:
+[Elab.definition.body] tacticRwa__ : Lean.ParserDescr :=
+Lean.ParserDescr.node `tacticRwa__ 1022
+  (Lean.ParserDescr.binary `andthen
+    (Lean.ParserDescr.binary `andthen (Lean.ParserDescr.nonReservedSymbol "rwa " false) Lean.Parser.Tactic.rwRuleSeq)
+    (Lean.ParserDescr.unary `optional Lean.Parser.Tactic.location))
+
+-/
+
+```
+
+Literals are written as double-quoted strings (`"rwa "` expects the literal
+sequence of characters `rwa`, while the trailing space provides a hint to the
+formatter that it should add a space after `rwa` when pretty printing this
+syntax); `rwRuleSeq` and `location` are themselves `ParserDescr`s, and we finish
+with `: tactic` specifying that the preceding parser is for an element in the
+`tactic` syntax category. The parentheses around `(location)?` are necessary
+(rather than `location?`) because Lean 4 allows question marks to be used in
+identifiers, so `location?` is one single identifier that ends with a question
+mark, which is not what we want.
+
+The name `tacticRwa__` is automatically generated. You can name parser
+descriptors declared with the `syntax` keyword like so:
+
+```lean
+set_option trace.Elab.definition true in
+syntax (name := introv) "introv " (colGt ident)* : tactic
+
+/-
+[Elab.definition.body] introv : Lean.ParserDescr :=
+Lean.ParserDescr.node `introv 1022
+  (Lean.ParserDescr.binary `andthen (Lean.ParserDescr.nonReservedSymbol "introv " false)
+    (Lean.ParserDescr.unary `many
+      (Lean.ParserDescr.binary `andthen (Lean.ParserDescr.const `colGt) (Lean.ParserDescr.const `ident))))
+-/
+```
+
+## The pattern language
+
+Available quantifiers are `?` (one or zero occurrences, see note below), `*`
+(zero or more occurrences), and `+` (one or more occurrences).
+
+Keep in mind that Lean makes `?` available for use in identifiers, so if we want
+a parser to look for an optional `location`, we would need to write
+`(location)?` with parenthesis acting as a separator, since `location?` would
+look for something under the identifier `location?` (where the `?` is part of
+the identifier).
+
+Parentheses can be used as delimiters.
+
+Separated lists can be constructed like so: `$ts,*` for a comma separated list.
+
+"extended splices" can be constructed as `$[..]`. See the official paper (p. 12)
+for more details.
+
+Literals are written as double-quoted strings. A literal may use trailing
+whitespace (see e.g. the `rwa` or `introv` tactics) to tell the pretty-printer
+how it should be displayed, but such whitespace will not prevent a literal with
+no trailing whitespace from matching. The spaces are relevant, but not
+interpreted literally. When the ParserDescr is turned into a Parser, the actual
+token matcher [uses the .trim of the provided
+string](https://github.com/leanprover/lean4/blob/53ec43ff9b8f55989b12c271e368287b7b997b54/src/Lean/Parser/Basic.lean#L1193),
+but the generated formatter [uses the spaces as
+specified](https://github.com/leanprover/lean4/blob/8d370f151f7c88a687152a5b161dcb484c446ce2/src/Lean/PrettyPrinter/Formatter.lean#L328),
+that is, turning the atom "rwa" in the syntax into the string rwa as part of the
+pretty printed output.
+
+## Syntax expansions with `macro_rules`, and how it desugars.
+
+`macro_rules` lets you declare expansions for a given `Syntax` element using a
+syntax similar to a `match` statement. The left-hand side of a match arm is a
+quotation (with a leading `<cat>|` for categories other than `term` and
+`command`) in which users can specify the pattern they'd like to write an
+expansion for. The right-hand side returns a syntax quotation which is the
+output the user wants to expand to.
+
+A feature of Lean's macro system is that if there are multiple expansions for a
+particular match, Lean will try the most recently declared expansion first, and
+will retry with other matching expansions if the previous attempt failed. This
+is particularly useful for extending existing tactics.
+
+The following example shows both the retry behavior, and the fact that macros
+declared using the shorthand `macro` syntax can still have additional expansions
+declared with `macro_rules`. This `transitivity` tactic is implemented such that
+it will work for either Nat.le or Nat.lt. The Nat.lt version was declared "most
+recently", so it will be tried first, but if it fails (for example, if the
+actual term in question is Nat.le) the next potential expansion will be tried:
+```lean
+macro "transitivity" e:(colGt term) : tactic => `(tactic| apply Nat.le_trans (m := $e))
+macro_rules
+  | `(tactic| transitivity $e) => `(tactic| apply Nat.lt_trans (m := $e))
+
+example (a b c : Nat) (h0 : a < b) (h1 : b < c) : a < c := by
+  transitivity b <;>
+  assumption
+
+example (a b c : Nat) (h0 : a <= b) (h1 : b <= c) : a <= c := by
+  transitivity b <;>
+  assumption
+
+/- This will fail, but is interesting in that it exposes the "most-recent first" behavior, since the
+  error message complains about being unable to unify mvar1 <= mvar2, rather than mvar1 < mvar2. -/
+/-
+example (a b c : Nat) (h0 : a <= b) (h1 : b <= c) : False := by
+  transitivity b <;>
+  assumption
+-/
+```
+
+To see the desugared definition of the actual expansion, we can again use
+`set_option trace.Elab.definition true in` and observe the output of the humble
+`exfalso` tactic defined in Mathlib4:
+```lean
+
+set_option trace.Elab.definition true in
+macro "exfalso" : tactic => `(tactic| apply False.elim)
+
+/-
+Results in the expansion:
+
+[Elab.definition.body] _aux___macroRules_tacticExfalso_1 : Lean.Macro :=
+fun x =>
+  let discr := x;
+  /- This is where Lean tries to actually identify that it's an invocation of the exfalso tactic -/
+  if Lean.Syntax.isOfKind discr `tacticExfalso = true then
+    let discr := Lean.Syntax.getArg discr 0;
+    let x := discr;
+    do
+      /- Lean getting scope/meta info from the macro monad -/
+      let info ← Lean.MonadRef.mkInfoFromRefPos
+      let scp ← Lean.getCurrMacroScope
+      let mainModule ← Lean.getMainModule
+      pure
+          (Lean.Syntax.node Lean.SourceInfo.none `Lean.Parser.Tactic.seq1
+            #[Lean.Syntax.node Lean.SourceInfo.none `null
+                #[Lean.Syntax.node Lean.SourceInfo.none `Lean.Parser.Tactic.apply
+                    #[Lean.Syntax.atom info "apply",
+                      Lean.Syntax.ident info (String.toSubstring "False.elim")
+                        (Lean.addMacroScope mainModule `False.elim scp) [(`False.elim, [])]]]])
+  else
+    /- If this wasn't actually an invocation of the exfalso tactic, throw the "unsupportedSyntax" error -/
+    let discr := x;
+    throw Lean.Macro.Exception.unsupportedSyntax
+-/
+```
+
+We can also create the syntax transformer declaration ourselves instead of using
+`macro_rules`. We'll need to name our parser and use the attribute `@[macro
+myExFalsoParser]` to associate our declaration with the parser:
+```lean
+# open Lean
+syntax (name := myExfalsoParser) "myExfalso" : tactic
+
+-- remember that `Macro` is a synonym for `Syntax -> TacticM Unit`
+@[macro myExfalsoParser] def implMyExfalso : Macro :=
+fun stx => `(tactic| apply False.elim)
+
+example (p : Prop) (h : p) (f : p -> False) : 3 = 2 := by
+  myExfalso
+  exact f h
+```
+
+In the above example, we're still using the sugar Lean provides for creating
+quotations, as it feels more intuitive and saves us some work. It is possible to
+forego the sugar altogether:
+```lean
+syntax (name := myExfalsoParser) "myExfalso" : tactic
+
+@[macro myExfalsoParser] def implMyExfalso : Lean.Macro :=
+  fun stx => pure (Lean.mkNode `Lean.Parser.Tactic.apply
+    #[Lean.mkAtomFrom stx "apply", Lean.mkCIdentFrom stx ``False.elim])
+
+example (p : Prop) (h : p) (f : p -> False) : 3 = 2 := by
+  myExfalso
+  exact f h
+```
+
+## The `macro` keyword
+
+`macro` is a shortcut which allows users to declare both a parser and an
+expansion at the same time as a matter of convenience. Additional expansions for
+the parser generated by the `macro` invocation can be added with a separate
+`macro_rules` block (see the example in the `macro_rules` section).
+
+## Unexpanders
+
+TODO; for now, see the unexpander in Mathlib.Set for an example.
+
+## More illustrative examples:
+
+The
+[Tactic.Basic](https://github.com/leanprover-community/mathlib4/blob/master/Mathlib/Tactic/Basic.lean)
+file in Mathlib4 contains many good examples to learn from.
+
+## Practical tips:
+
+You can observe the output of commands and functions that in some way use the
+macro system by setting this option to true : `set_option trace.Elab.definition
+true`
+
+Lean also offers the option of limiting the region in which option is set with
+the syntax `set_option ... in`):
+
+Hygiene can be disabled with the command option `set_option hygiene false`

doc/make/index.md

@@ -1,6 +1,6 @@
-These are instructions to set up a working development environment for those who wish to make changes to Lean itself. It is part of the [Development Guide](../dev/index.md).
+These are instructions to set up a working development environment for those who wish to make changes to Lean itself. It is part of the [Development Guide](doc/dev/index.md).
 
-We strongly suggest that new users instead follow the [Quickstart](../quickstart.md) to get started using Lean, since this sets up an environment that can automatically manage multiple Lean toolchain versions, which is necessary when working within the Lean ecosystem.
+We strongly suggest that new users instead follow the [Quickstart](doc/quickstart.md) to get started using Lean, since this sets up an environment that can automatically manage multiple Lean toolchain versions, which is necessary when working within the Lean ecosystem.
 
 Requirements
 ------------
@@ -44,20 +44,15 @@ Useful CMake Configuration Settings
 Pass these along with the `cmake --preset release` command.
 There are also two alternative presets that combine some of these options you can use instead of `release`: `debug` and `sandebug` (sanitize + debug).
 
-* `-DCMAKE_BUILD_TYPE=`\
+* `-D CMAKE_BUILD_TYPE=`\
   Select the build type. Valid values are `RELEASE` (default), `DEBUG`,
   `RELWITHDEBINFO`, and `MINSIZEREL`.
 
-* `-DCMAKE_C_COMPILER=`\
-  `-DCMAKE_CXX_COMPILER=`\
+* `-D CMAKE_C_COMPILER=`\
+  `-D CMAKE_CXX_COMPILER=`\
   Select the C/C++ compilers to use. Official Lean releases currently use Clang;
   see also `.github/workflows/ci.yml` for the CI config.
 
-* `-DUSE_LAKE=ON`\
-  Experimental option to build the core libraries using Lake instead of `lean.mk`.  Caveats:
-  * As native code compilation is still handled by cmake, changes to stage0/ (such as from `git pull`) are picked up only when invoking the build via `make`, not via `Refresh Dependencies` in the editor.
-  * `USE_LAKE` is not yet compatible with `LAKE_ARTIFACT_CACHE`
-
 Lean will automatically use [CCache](https://ccache.dev/) if available to avoid
 redundant builds, especially after stage 0 has been updated.
 

doc/make/msys2.md

@@ -15,24 +15,17 @@ Mode](https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-devi
 which will allow Lean to create symlinks that e.g. enable go-to-definition in
 the stdlib.
 
-## Installing the Windows SDK
-
-Install the Windows SDK from [Microsoft](https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/).
-The oldest supported version is 10.0.18362.0. If you installed the Windows SDK to the default location,
-then there should be a directory with the version number at `C:\Program Files (x86)\Windows Kits\10\Include`.
-If there are multiple directories, only the highest version number matters.
-
 ## Installing dependencies
 
 [The official webpage of MSYS2][msys2] provides one-click installers.
-Once installed, you should run the "MSYS2 CLANG64" shell from the start menu (the one that runs `clang64.exe`).
-Do not run "MSYS2 MSYS" or "MSYS2 MINGW64" instead!
-MSYS2 has a package management system, [pacman][pacman].
+Once installed, you should run the "MSYS2 MinGW 64-bit shell" from the start menu (the one that runs `mingw64.exe`).
+Do not run "MSYS2 MSYS" instead!
+MSYS2 has a package management system, [pacman][pacman], which is used in Arch Linux.
 
 Here are the commands to install all dependencies needed to compile Lean on your machine.
 
 ```bash
-pacman -S make python mingw-w64-clang-x86_64-cmake mingw-w64-clang-x86_64-clang mingw-w64-clang-x86_64-ccache mingw-w64-clang-x86_64-libuv mingw-w64-clang-x86_64-gmp git unzip diffutils binutils
+pacman -S make python mingw-w64-x86_64-cmake mingw-w64-x86_64-clang mingw-w64-x86_64-ccache mingw-w64-x86_64-libuv mingw-w64-x86_64-gmp git unzip diffutils binutils
 ```
 
 You should now be able to run these commands:
@@ -68,7 +61,8 @@ If you want a version that can run independently of your MSYS install
 then you need to copy the following dependent DLL's from where ever
 they are installed in your MSYS setup:
 
-- libc++.dll
+- libgcc_s_seh-1.dll
+- libstdc++-6.dll
 - libgmp-10.dll
 - libuv-1.dll
 - libwinpthread-1.dll
@@ -88,6 +82,6 @@ version clang to your path.
 
 **-bash: gcc: command not found**
 
-Make sure `/clang64/bin` is in your PATH environment.  If it is not then
-check you launched the MSYS2 CLANG64 shell from the start menu.
-(The one that runs `clang64.exe`).
+Make sure `/mingw64/bin` is in your PATH environment.  If it is not then
+check you launched the MSYS2 MinGW 64-bit shell from the start menu.
+(The one that runs `mingw64.exe`).

doc/make/osx-10.9.md

@@ -1,4 +1,4 @@
-# Install Packages on OS X
+# Install Packages on OS X 14.5
 
 We assume that you are using [homebrew][homebrew] as a package manager.
 
@@ -6,23 +6,23 @@ We assume that you are using [homebrew][homebrew] as a package manager.
 
 ## Compilers
 
-You need a C++14-compatible compiler to build Lean. As of July
-2025, you have three options:
+You need a C++11-compatible compiler to build Lean. As of November
+2014, you have three options:
 
- - clang++ shipped with OSX (at time of writing v17.0.0)
- - clang++ via homebrew (at time of writing, v20.1.8)
- - gcc via homebrew (at time of writing, v15.1.0)
+ - clang++-3.5 (shipped with OSX, Apple LLVM version 6.0)
+ - gcc-4.9.1 (homebrew)
+ - clang++-3.5 (homebrew)
 
 We recommend to use Apple's clang++ because it is pre-shipped with OS
 X and requires no further installation.
 
-To install gcc via homebrew, please execute:
+To install gcc-4.9.1 via homebrew, please execute:
 ```bash
 brew install gcc
 ```
-To install clang via homebrew, please execute:
+To install clang++-3.5 via homebrew, please execute:
 ```bash
-brew install llvm lld
+brew install llvm
 ```
 To use compilers other than the default one (Apple's clang++), you
 need to use `-DCMAKE_CXX_COMPILER` option to specify the compiler
@@ -32,13 +32,12 @@ following to use `g++`.
 cmake -DCMAKE_CXX_COMPILER=g++ ...
 ```
 
-## Required Packages: CMake, GMP, libuv, pkgconf
+## Required Packages: CMake, GMP, libuv
 
 ```bash
 brew install cmake
 brew install gmp
 brew install libuv
-brew install pkgconf
 ```
 
 ## Recommended Packages: CCache

doc/make/ubuntu.md

@@ -8,5 +8,5 @@ follow the [generic build instructions](index.md).
 ## Basic packages
 
 ```bash
-sudo apt-get install git libgmp-dev libuv1-dev cmake ccache clang pkgconf
+sudo apt-get install git libgmp-dev libuv1-dev cmake ccache clang
 ```

doc/metaprogramming-arith.md

@@ -0,0 +1,134 @@
+# Arithmetic as an embedded domain-specific language
+
+Let's parse another classic grammar, the grammar of arithmetic expressions with
+addition, multiplication, integers, and variables.  In the process, we'll learn
+how to:
+
+- Convert identifiers such as `x` into strings within a macro.
+- add the ability to "escape" the macro context from within the macro. This is useful to interpret identifiers with their _original_ meaning (predefined values)
+  instead of their new meaning within a macro (treat as a symbol).
+
+Let's begin with the simplest thing possible. We'll define an AST, and use operators `+` and `*` to denote
+building an arithmetic AST.
+
+
+Here's the AST that we will be parsing:
+
+```lean,ignore
+{{#include metaprogramming-arith.lean:1:5}}
+```
+
+We declare a syntax category to describe the grammar that we will be parsing.
+See that we control the precedence of `+` and `*` by writing `syntax:50` for addition and `syntax:60` for multiplication,
+indicating that multiplication binds tighter than addition (higher the number, tighter the binding).
+This allows us to declare _precedence_ when defining new syntax.
+
+```lean,ignore
+{{#include metaprogramming-arith.lean:7:13}}
+```
+
+Further, if we look at `syntax:60  arith:60 "+" arith:61 : arith`, the
+precedence declarations at `arith:60 "+" arith:61` conveys that the left
+argument must have precedence at least `60` or greater, and the right argument
+must have precedence at least`61` or greater.  Note that this forces left
+associativity. To understand this, let's compare two hypothetical parses:
+
+```
+-- syntax:60  arith:60 "+" arith:61 : arith -- Arith.add
+-- a + b + c
+(a:60 + b:61):60 + c
+a + (b:60 + c:61):60
+```
+
+In the parse tree of `a + (b:60 + c:61):60`, we see that the right argument `(b + c)` is given the precedence `60`. However,
+the rule for addition expects the right argument to have a precedence of **at least** 61, as witnessed by the `arith:61` at
+the right-hand-side of `syntax:60 arith:60 "+" arith:61 : arith`. Thus, the rule `syntax:60  arith:60 "+" arith:61 : arith`
+ensures that addition is left associative.
+
+Since addition is declared arguments of precedence `60/61` and multiplication with `70/71`, this causes multiplication to bind
+tighter than addition. Once again, let's compare two hypothetical parses:
+
+```
+-- syntax:60  arith:60 "+" arith:61 : arith -- Arith.add
+-- syntax:70 arith:70 "*" arith:71 : arith -- Arith.mul
+-- a * b + c
+a * (b:60 + c:61):60
+(a:70 * b:71):70 + c
+```
+
+While parsing `a * (b + c)`, `(b + c)` is assigned a precedence `60` by the addition rule. However, multiplication expects
+the right argument to have precedence **at least** 71. Thus, this parse is invalid. In contrast, `(a * b) + c` assigns
+a precedence of `70` to `(a * b)`. This is compatible with addition which expects the left argument to have precedence
+**at least `60` ** (`70` is greater than `60`). Thus, the string `a * b + c` is parsed as `(a * b) + c`.
+For more details, please look at the [Lean manual on syntax extensions](./notation.md#notations-and-precedence).
+
+To go from strings into `Arith`, we define a macro to
+translate the syntax category `arith` into an `Arith` inductive value that
+lives in `term`:
+
+
+```lean,ignore
+{{#include metaprogramming-arith.lean:15:16}}
+```
+
+Our macro rules perform the "obvious" translation:
+
+```lean,ignore
+{{#include metaprogramming-arith.lean:18:23}}
+```
+
+And some examples:
+
+```lean,ignore
+{{#include metaprogramming-arith.lean:25:41}}
+```
+
+Writing variables as strings, such as `"x"`  gets old; wouldn't it be so much
+prettier if we could write `x * y`, and have the macro translate this into `Arith.mul (Arith.Symbol "x") (Arith.mul "y")`?
+We can do this, and this will be our first taste of manipulating macro variables --- we'll use `x.getId` instead of directly evaluating `$x`.
+We also write a macro rule for `Arith|` that translates an identifier into
+a string, using `$(Lean.quote (toString x.getId))`:
+
+```lean,ignore
+{{#include metaprogramming-arith.lean:43:46}}
+```
+
+Let's test and see that we can now write expressions such as `x * y` directly instead of having to write `"x" * "y"`:
+
+```lean,ignore
+{{#include metaprogramming-arith.lean:48:51}}
+```
+
+We now show an unfortunate consequence of the above definitions. Suppose we want to build `(x + y) + z`.
+Since we already have defined `xPlusY` as `x + y`, perhaps we should reuse it! Let's try:
+
+```lean,ignore
+#check `[Arith| xPlusY + z]  -- Arith.add (Arith.symbol "xPlusY") (Arith.symbol "z")
+```
+
+Whoops, that didn't work! What happened? Lean treats `xPlusY` _itself_ as an identifier! So we need to add some syntax
+to be able to "escape" the `Arith|` context. Let's use the syntax `<[ $e:term ]>` to mean: evaluate `$e` as a real term,
+not an identifier. The macro looks like follows:
+
+```lean,ignore
+{{#include metaprogramming-arith.lean:53:56}}
+```
+
+Let's try our previous example:
+
+```lean,ignore
+{{#include metaprogramming-arith.lean:58:58}}
+```
+
+Perfect!
+
+In this tutorial, we expanded on the previous tutorial to parse a more
+realistic grammar with multiple levels of precedence, how to parse identifiers directly
+within a macro, and how to provide an escape from within the macro context.
+
+#### Full code listing
+
+```lean
+{{#include metaprogramming-arith.lean}}
+```
+

doc/mission.md

@@ -0,0 +1,12 @@
+Mission
+=======
+
+Empower software developers to design, develop, and reason about programs.
+Empower mathematicians and scientists to design, develop, and reason about formal models.
+
+How
+---
+
+Lean is an efficient functional programming language based on dependent type theory.
+It is under heavy development, but it already generates very efficient code.
+It also has a powerful meta-programming framework, extensible parser, and IDE support based on LSP.

doc/monads/.gitignore

@@ -0,0 +1 @@
+*.lean.md

doc/monads/applicatives.lean

@@ -0,0 +1,334 @@
+/-!
+# Applicative Functors
+
+Building on [Functors](functors.lean.md) is the [Applicative
+Functor](https://en.wikipedia.org/wiki/Applicative_functor). For simplicity, you can refer to these
+simply as "Applicatives". These are a little tricker than functors, but still simpler than monads.
+Let's see how they work!
+
+## What is an Applicative Functor?
+
+An applicative functor defines a default or "base" construction for an object and allows
+function application to be chained across multiple instances of the structure. All applicative
+functors are functors, meaning they must also support the "map" operation.
+
+## How are Applicatives represented in Lean?
+
+An [applicative functor](https://en.wikipedia.org/wiki/Applicative_functor) is an intermediate
+structure between `Functor` and `Monad`. It mainly consists of two operations:
+
+* `pure : α → F α`
+* `seq : F (α → β) → F α → F β` (written as `<*>`)
+
+The `pure` operator specifies how you can wrap a normal object `α` into an instance of this structure `F α`.
+This is the "default" mechanism mentioned above.
+
+The `seq` operator allows you to chain operations by wrapping a function in a structure. The name
+"applicative" comes from the fact that you "apply" functions from within the structure, rather than
+simply from outside the structure, as was the case with `Functor.map`.
+
+Applicative in Lean is built on some helper type classes, `Functor`, `Pure` and `Seq`:
+
+-/
+namespace hidden -- hidden
+class Applicative (f : Type u → Type v) extends Functor f, Pure f, Seq f, SeqLeft f, SeqRight f where
+  map      := fun x y => Seq.seq (pure x) fun _ => y
+  seqLeft  := fun a b => Seq.seq (Functor.map (Function.const _) a) b
+  seqRight := fun a b => Seq.seq (Functor.map (Function.const _ id) a) b
+end hidden -- hidden
+/-!
+Notice that as with `Functor` it is also a type transformer `(f : Type u → Type v)` and notice the
+`extends Functor f` is ensuring the base `Functor` also performs that same type transformation.
+
+As stated above, all applicatives are then functors. This means you can assume that `map` already
+exists for all these types.
+
+The `Pure` base type class is a very simple type class that supplies the `pure` function.
+
+-/
+namespace hidden -- hidden
+class Pure (f : Type u → Type v) where
+   pure {α : Type u} : α → f α
+end hidden -- hidden
+/-!
+
+You can think of it as lifting the result of a pure value to some monadic type. The simplest example
+of `pure` is the `Option` type:
+
+-/
+#eval (pure 10 : Option Nat)  -- some 10
+/-!
+
+Here we used the `Option` implementation of `pure` to wrap the `Nat 10` value in an `Option Nat`
+type resulting in the value `some 10`, and in fact if you look at the Monad instance of `Option` , you
+will see that `pure` is indeed implemented using `Option.some`:
+
+-/
+instance : Monad Option where
+    pure := Option.some
+/-!
+
+The `Seq` type class is also a simple type class that provides the `seq` operator which can
+also be written using the special syntax `<*>`.
+
+-/
+namespace hidden -- hidden
+class Seq (f : Type u → Type v) : Type (max (u+1) v) where
+  seq : {α β : Type u} → f (α → β) → (Unit → f α) → f β
+end hidden -- hidden
+/-!
+
+
+## Basic Applicative Examples
+
+Many of the basic functors also have instances of `Applicative`.
+For example, `Option` is also `Applicative`.
+
+So let's take a look and what the `seq` operator can do.  Suppose you want to multiply two `Option Nat`
+objects.  Your first attempt might be this:
+
+-/
+#check_failure (some 4) * (some 5)      -- failed to synthesize instance
+/-!
+
+You then might wonder how to use the `Functor.map` to solve this since you could do these before:
+
+-/
+#eval (some 4).map (fun x => x * 5)  -- some 20
+
+#eval (some 4).map (· * 5)  -- some 20
+
+#eval (· * 5) <$> (some 4)   -- some 20
+/-!
+
+Remember that `<$>` is the infix notation for `Functor.map`.
+
+The functor `map` operation can apply a multiplication to the value in the `Option` and then lift the
+result back up to become a new `Option` , but this isn't what you need here.
+
+The `Seq.seq` operator `<*>` can help since it can apply a function to the items inside a
+container and then lift the result back up to the desired type, namely `Option` .
+
+There are two ways to do this:
+
+-/
+#eval pure (.*.) <*> some 4 <*> some 5 -- some 20
+
+#eval (.*.) <$> some 4 <*> some 5 -- some 20
+/-!
+
+In the first way, we start off by wrapping the function in an applicative using pure. Then we apply
+this to the first `Option` , and again to the second `Option`  in a chain of operations.  So  you can see
+how `Seq.seq` can be chained in fact, `Seq.seq` is really all about chaining of operations.
+
+But in this case there is a simpler way.  In the second way, you can see that "applying" a single
+function to a container is the same as using `Functor.map`. So you use `<$>` to "transform" the first
+option into an `Option` containing a function, and then apply this function over the second value.
+
+Now if either side is `none`, the result is `none`, as expected, and in this case the
+`seq` operator was able to eliminate the multiplication:
+
+-/
+#eval (.*.) <$> none <*> some 5  -- none
+#eval (.*.) <$> some 4 <*> none  -- none
+/-!
+
+For a more interesting example, let's make `List` an applicative by adding the following
+definition:
+
+-/
+instance : Applicative List where
+  pure := List.pure
+  seq f x := List.bind f fun y => Functor.map y (x ())
+/-!
+
+Notice you can now sequence a _list_ of functions and a _list_ of items.
+The trivial case of sequencing a singleton list is in fact the same as `map`, as you saw
+earlier with the `Option`  examples:
+
+-/
+#eval [ (·+2)] <*> [4, 6] -- [6, 8]
+#eval (·+2) <$> [4,6]    -- [6, 8]
+/-!
+
+But now with list it is easier to show the difference when you do this:
+
+-/
+#eval [(·+2), (· *3)] <*> [4, 6] -- [6, 8, 12, 18]
+/-!
+
+Why did this produce 4 values?  The reason is because `<*>` applies _every_ function to _every_
+value in a pairwise manner.  This makes sequence really convenient for solving certain problems. For
+example, how do you get the pairwise combinations of all values from two lists?
+
+-/
+#eval Prod.mk <$> [1, 2, 3] <*> [4, 5, 6]
+-- [(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)]
+/-!
+
+How do you get the sum of these pairwise values?
+-/
+#eval (·+·) <$> [1, 2, 3] <*> [4, 5, 6]
+-- [5, 6, 7, 6, 7, 8, 7, 8, 9]
+/-!
+
+Here you can use `<$>` to "transform" each element of the first list into a function, and then apply
+these functions over the second list.
+
+If you have 3 lists, and want to find all combinations of 3 values across those lists you
+would need helper function that can create a tuple out of 3 values, and Lean provides a
+very convenient syntax for that `(·,·,·)`:
+
+-/
+#eval (·,·,·) <$> [1, 2] <*> [3, 4] <*> [5, 6]
+-- [(1, 3, 5), (1, 3, 6), (1, 4, 5), (1, 4, 6), (2, 3, 5), (2, 3, 6), (2, 4, 5), (2, 4, 6)]
+/-!
+
+And you could sum these combinations if you first define a sum function that takes three inputs and
+then you could chain apply this over the three lists.  Again lean can create such a function
+with the expression `(·+·+·)`:
+
+-/
+#eval (·+·+·) <$> [1, 2] <*> [3, 4] <*> [5, 6]
+-- [9, 10, 10, 11, 10, 11, 11, 12]
+/-!
+
+And indeed each sum here matches the expected values if you manually sum the triples we
+show above.
+
+**Side note:** there is another way to combine lists with a function that does not do the pairwise
+combinatorics, it is called `List.zipWith`:
+
+-/
+#eval List.zipWith (·+·) [1, 2, 3] [4, 5, 6]
+-- [5, 7, 9]
+/-!
+
+And there is a helper function named `List.zip` that calls `zipWith` using the function `Prod.mk`
+so you get a nice zipped list like this:
+
+-/
+#eval List.zip [1, 2, 3] [4, 5, 6]
+-- [(1, 4), (2, 5), (3, 6)]
+/-!
+
+And of course, as you would expect, there is an `unzip` also:
+
+-/
+#eval List.unzip (List.zip [1, 2, 3] [4, 5, 6])
+-- ([1, 2, 3], [4, 5, 6])
+/-!
+
+## Example: A Functor that is not Applicative
+
+From the chapter on [functors](functors.lean.md) you might remember this example of `LivingSpace`
+that had a `Functor` instance:
+
+-/
+structure LivingSpace (α : Type) where
+  totalSize : α
+  numBedrooms : Nat
+  masterBedroomSize : α
+  livingRoomSize : α
+  kitchenSize : α
+  deriving Repr, BEq
+
+def LivingSpace.map (f : α → β) (s : LivingSpace α) : LivingSpace β :=
+  { totalSize := f s.totalSize
+    numBedrooms := s.numBedrooms
+    masterBedroomSize := f s.masterBedroomSize
+    livingRoomSize := f s.livingRoomSize
+    kitchenSize := f s.kitchenSize }
+
+instance : Functor LivingSpace where
+  map := LivingSpace.map
+/-!
+
+It wouldn't really make sense to make an `Applicative` instance here. How would you write `pure` in
+the `Applicative` instance? By taking a single value and plugging it in for total size _and_ the
+master bedroom size _and_ the living room size? That wouldn't really make sense. And what would the
+numBedrooms value be for the default? What would it mean to "chain" two of these objects together?
+
+If you can't answer these questions very well, then it suggests this type isn't really an
+Applicative functor.
+
+## SeqLeft and SeqRight
+
+You may remember seeing the `SeqLeft` and `SeqRight` base types on `class Applicative` earlier.
+These provide the `seqLeft` and `seqRight` operations which also have some handy notation
+shorthands `<*` and `*>` respectively. Where: `x <* y` evaluates `x`, then `y`, and returns the
+result of `x` and `x *> y` evaluates `x`, then `y`,  and returns the result of `y`.
+
+To make it easier to remember, notice that it returns that value that the `<*` or `*>` notation is
+pointing at.  For example:
+
+-/
+#eval (some 1) *> (some 2) -- Some 2
+#eval (some 1) <* (some 2) -- Some 1
+/-!
+
+So these are a kind of "discard" operation.  Run all the actions, but only return the values that you
+care about. It will be easier to see these in action when you get to full Monads, but they are used
+heavily in the Lean `Parsec` parser combinator library where you will find parsing functions like
+this one which parses the XML declaration `<?xml version="1.0" encoding='utf-8' standalone="yes">`:
+
+```lean
+def XMLdecl : Parsec Unit := do
+  skipString "<?xml"
+  VersionInfo
+  optional EncodingDecl *> optional SDDecl *> optional S *> skipString "?>"
+```
+
+But you will need to understand full Monads before this will make sense.
+
+## Lazy Evaluation
+
+Diving a bit deeper, (you can skip this and jump to the [Applicative
+Laws](laws.lean.md#what-are-the-applicative-laws) if don't want to dive into this implementation detail right
+now). But, if you write a simple `Option` example `(.*.) <$> some 4 <*> some 5` that produces `some 20`
+using `Seq.seq` you will see something interesting:
+
+-/
+#eval Seq.seq ((.*.) <$> some 4) (fun (_ : Unit) => some 5) -- some 20
+/-!
+
+This may look a bit cumbersome, specifically, why did we need to invent this funny looking function
+`fun (_ : Unit) => (some 5)`?
+
+Well if you take a close look at the type class definition:
+```lean
+class Seq (f : Type u → Type v) where
+  seq : {α β : Type u} → f (α → β) → (Unit → f α) → f β
+```
+
+You will see this function defined here: `(Unit → f α)`, this is a function that takes `Unit` as input
+and produces the output of type `f α` where `f` is the container type `Type u -> Type v`, in this example `Option`
+and `α` is the element type `Nat`, so `fun (_ : Unit) => some 5` matches this definition because
+it is taking an input of type Unit and producing `some 5` which is type `Option Nat`.
+
+The that `seq` is defined this way is because Lean is an eagerly evaluated language
+(call-by-value), you have to use this kind of Unit function whenever you want to explicitly delay
+evaluation and `seq` wants that so it can eliminate unnecessary function evaluations whenever
+possible.
+
+Fortunately the `<*>` infix notation hides this from you by creating this wrapper function for you.
+If you look up the notation using F12 in VS Code you will find it contains `(fun _ : Unit => b)`.
+
+Now to complete this picture you will find the default implementation of `seq` on the Lean `Monad`
+type class:
+
+```lean
+class Monad (m : Type u → Type v) extends Applicative m, Bind m where
+  seq      f x := bind f fun y => Functor.map y (x ())
+```
+
+Notice here that `x` is the `(Unit → f α)` function, and it is calling that function by passing the
+Unit value `()`, which is the Unit value (Unit.unit).  All this just to ensure delayed evaluation.
+
+## How do Applicatives help with Monads?
+
+Applicatives are helpful for the same reasons as functors. They're a relatively simple abstract
+structure that has practical applications in your code. Now that you understand how chaining
+operations can fit into a structure definition, you're in a good position to start learning about
+[Monads](monads.lean.md)!
+-/

doc/monads/except.lean

@@ -0,0 +1,178 @@
+/-!
+# Except
+
+The `Except` Monad adds exception handling behavior to your functions.  Exception handling
+in other languages like Python or Java is done with a built in `throw` method that you
+can use anywhere.  In `Lean` you can only `throw` an exception when your function is
+executing in the context of an `Except` monad.
+
+-/
+def divide (x y: Float): Except String Float :=
+  if y == 0 then
+    throw "can't divide by zero"
+  else
+    pure (x / y)
+
+#eval divide 5 2  -- Except.ok 2.500000
+#eval divide 5 0  -- Except.error "can't divide by zero"
+
+/-!
+
+Just as the `read` operation was available from the `ReaderM` monad and the `get` and `set`
+operations came with the `StateM` monad, here you can see a `throw` operation is provided by the
+`Except` monad.
+
+So in Lean, `throw` is not available everywhere like it is in most imperative programming languages.
+You have to declare your function can throw by changing the type signature to `Except String Float`.
+This creates a function that might return an error of type `String` or it might return a value of
+type `Float` in the non-error case.
+
+Once your function is monadic you also need to use the `pure` constructor of the `Except` monad to
+convert the pure non-monadic value `x / y` into the required `Except` object.  See
+[Applicatives](applicatives.lean.md) for details on `pure`.
+
+Now this return typing would get tedious if you had to include it everywhere that you call this
+function, however, Lean type inference can clean this up. For example, you can define a test
+function that calls the `divide` function and you don't need to say anything here about the fact that
+it might throw an error, because that is inferred:
+-/
+def test := divide 5 0
+
+#check test     -- Except String Float
+/-!
+
+Notice the Lean compiler infers the required `Except String Float` type information for you.
+And now you can run this test and get the expected exception:
+
+-/
+#eval test      -- Except.error "can't divide by zero"
+/-!
+
+
+## Chaining
+
+Now as before you can build a chain of monadic actions that can be composed together using `bind (>>=)`:
+-/
+def square (x : Float) : Except String Float :=
+  if x >= 100 then
+    throw "it's absolutely huge"
+  else
+    pure (x * x)
+
+#eval divide 6 2 >>= square  -- Except.ok 9.000000
+#eval divide 6 0 >>= square  -- Except.error "can't divide by zero"
+#eval divide 100 1 >>= square  -- Except.error "it's absolutely huge"
+
+def chainUsingDoNotation := do
+  let r ← divide 6 0
+  square r
+
+#eval chainUsingDoNotation  -- Except.error "can't divide by zero"
+
+/-!
+Notice in the second `divide 6 0` the exception from that division was nicely propagated along
+to the final result and the square function was ignored in that case.  You can see why the
+`square` function was ignored if you look at the implementation of `Except.bind`:
+-/
+def bind (ma : Except ε α) (f : α → Except ε β) : Except ε β :=
+  match ma with
+  | Except.error err => Except.error err
+  | Except.ok v      => f v
+/-!
+
+Specifically notice that it only calls the next function `f v` in the `Except.ok`, and
+in the error case it simply passes the same error along.
+
+Remember also that you can chain the actions with implicit binding by using the `do` notation
+as you see in the `chainUsingDoNotation` function above.
+
+## Try/Catch
+
+Now with all good exception handling you also want to be able to catch exceptions so your program
+can continue on or do some error recovery task, which you can do like this:
+-/
+def testCatch :=
+  try
+    let r ← divide 8 0  -- 'r' is type Float
+    pure (toString r)
+  catch e =>
+    pure s!"Caught exception: {e}"
+
+#check testCatch -- Except String String
+/-!
+
+Note that the type inferred by Lean for this function is `Except String String` so unlike the
+`test` function earlier, this time Lean type inference has figured out that since the pure
+value `(toString r)` is of type `String`, then this function must have type `Except String String`
+so you don't have to explicitly state this. You can always hover your mouse over `testCatch`
+or use `#check testCatch` to query Lean interactively to figure out what type inference
+has decided.  Lean type inference makes life easy for you, so it's good to use it
+when you can.
+
+You can now see the try/catch working in this eval:
+-/
+#eval testCatch -- Except.ok "Caught exception: can't divide by zero"
+/-!
+
+Notice the `Caught exception:` wrapped message is returned, and that it is returned as an
+`Except.ok` value, meaning `testCatch` eliminated the error result as expected.
+
+So you've interleaved a new concept into your functions (exception handling) and the compiler is still
+able to type check everything just as well as it does for pure functions and it's been able to infer
+some things along the way to make it even easier to manage.
+
+Now you might be wondering why `testCatch` doesn't infer the return type `String`? Lean does this as a
+convenience since you could have a rethrow in or after the catch block. If you really want to stop
+the `Except` type from bubbling up you can unwrap it like this:
+
+-/
+def testUnwrap : String := Id.run do
+  let r ← divide 8 0 -- r is type Except String Float
+  match r with
+  | .ok a => toString a -- 'a' is type Float
+  | .error e => s!"Caught exception: {e}"
+
+#check testUnwrap -- String
+#eval testUnwrap -- "Caught exception: can't divide by zero"
+
+/-!
+
+The `Id.run` function is a helper function that executes the `do` block and returns the result where
+`Id` is the _identity monad_.  So `Id.run do` is a pattern you can use to execute monads in a
+function that is not itself monadic.  This works for all monads except `IO` which, as stated earlier,
+you cannot invent out of thin air, you must use the `IO` monad given to your `main` function.
+
+## Monadic functions
+
+You can also write functions that are designed to operate in the context of a monad.
+These functions typically end in upper case M like `List.forM` used below:
+-/
+
+def validateList (x : List Nat) (max : Nat): Except String Unit := do
+  x.forM fun a => do
+    if a > max then throw "illegal value found in list"
+
+#eval validateList [1, 2, 5, 3, 8] 10 -- Except.ok ()
+#eval validateList [1, 2, 5, 3, 8] 5 -- Except.error "illegal value found in list"
+
+/-!
+Notice here that the `List.forM` function passes the monadic context through to the inner function
+so it can use the `throw` function from the `Except` monad.
+
+The `List.forM` function is defined like this where `[Monad m]` means "in the context of a monad `m`":
+
+-/
+def forM [Monad m] (as : List α) (f : α → m PUnit) : m PUnit :=
+  match as with
+  | []      => pure ⟨⟩
+  | a :: as => do f a; List.forM as f
+/-!
+
+## Summary
+
+Now that you know all these different monad constructs, you might be wondering how you can combine
+them. What if there was some part of your state that you wanted to be able to modify (using the
+State monad), but you also needed exception handling. How can you get multiple monadic capabilities
+in the same function. To learn the answer, head to [Monad Transformers](transformers.lean.md).
+
+-/

doc/monads/functors.lean

@@ -0,0 +1,227 @@
+/-!
+# Functor
+
+A `Functor` is any type that can act as a generic container that allows you to transform the
+underlying values inside the container using a function, so that the values are all updated, but the
+structure of the container is the same. This is called "mapping".
+
+A List is one of the most basic examples of a `Functor`.
+
+A list contains zero or more elements of the same, underlying type.  When you `map` a function over
+a list, you create a new list with the same number of elements, where each has been transformed by
+the function:
+-/
+#eval List.map (λ x => toString x) [1,2,3] -- ["1", "2", "3"]
+
+-- you can also write this using dot notation on the List object
+#eval [1,2,3].map (λ x => toString x)  -- ["1", "2", "3"]
+
+/-!
+Here we converted a list of natural numbers (Nat) to a list of strings where the lambda function
+here used `toString` to do the transformation of each element. Notice that when you apply `map` the
+"structure" of the object remains the same, in this case the result is always a `List` of the same
+size.
+
+Note that in Lean a lambda function can be written using `fun` keyword or the unicode
+symbol `λ` which you can type in VS code using `\la `.
+
+List has a specialized version of `map` defined as follows:
+-/
+def map (f : α → β) : List α → List β
+  | []    => []
+  | a::as => f a :: map f as
+
+/-!
+This is a very generic `map` function that can take any function that converts `(α → β)` and use it
+to convert `List α → List β`. Notice the function call `f a` above, this application of `f` is
+producing the converted items for the new list.
+
+Let's look at some more examples:
+
+-/
+-- List String → List Nat
+#eval ["elephant", "tiger", "giraffe"].map (fun s => s.length)
+-- [8, 5, 7]
+
+-- List Nat → List Float
+#eval [1,2,3,4,5].map (fun s => (s.toFloat) ^ 3.0)
+-- [1.000000, 8.000000, 27.000000, 64.000000, 125.000000]
+
+--- List String → List String
+#eval ["chris", "david", "mark"].map (fun s => s.capitalize)
+-- ["Chris", "David", "Mark"]
+/-!
+
+Another example of a functor is the `Option` type. Option contains a value or nothing and is handy
+for code that has to deal with optional values, like optional command line arguments.
+
+Remember you can construct an Option using the type constructors `some` or `none`:
+
+-/
+#check some 5 -- Option Nat
+#eval some 5  -- some 5
+#eval (some 5).map (fun x => x + 1) -- some 6
+#eval (some 5).map (fun x => toString x) -- some "5"
+/-!
+
+Lean also provides a convenient short hand syntax for `(fun x => x + 1)`, namely `(· + 1)`
+using the middle dot unicode character which you can type in VS code using `\. `.
+
+-/
+#eval (some 4).map (· * 5)  -- some 20
+/-!
+
+The `map` function preserves the `none` state of the Option, so again
+map preserves the structure of the object.
+
+-/
+def x : Option Nat := none
+#eval x.map (fun x => toString x) -- none
+#check x.map (fun x => toString x) -- Option String
+/-!
+
+Notice that even in the `none` case it has transformed `Option Nat` into `Option String` as
+you see in the `#check` command.
+
+## How to make a Functor Instance?
+
+The `List` type is made an official `Functor` by the following type class instance:
+
+-/
+instance : Functor List where
+  map := List.map
+/-!
+
+Notice all you need to do is provide the `map` function implementation.  For a quick
+example, let's supposed you create a new type describing the measurements of a home
+or apartment:
+
+-/
+structure LivingSpace (α : Type) where
+  totalSize : α
+  numBedrooms : Nat
+  masterBedroomSize : α
+  livingRoomSize : α
+  kitchenSize : α
+  deriving Repr, BEq
+/-!
+
+Now you can construct a `LivingSpace` in square feet using floating point values:
+-/
+abbrev SquareFeet := Float
+
+def mySpace : LivingSpace SquareFeet :=
+  { totalSize := 1800, numBedrooms := 4, masterBedroomSize := 500,
+    livingRoomSize := 900, kitchenSize := 400 }
+/-!
+
+Now, suppose you want anyone to be able to map a `LivingSpace` from one type of measurement unit to
+another.  Then you would provide a `Functor` instance as follows:
+
+-/
+def LivingSpace.map (f : α → β) (s : LivingSpace α) : LivingSpace β :=
+  { totalSize := f s.totalSize
+    numBedrooms := s.numBedrooms
+    masterBedroomSize := f s.masterBedroomSize
+    livingRoomSize := f s.livingRoomSize
+    kitchenSize := f s.kitchenSize }
+
+instance : Functor LivingSpace where
+  map := LivingSpace.map
+/-!
+
+Notice this functor instance takes `LivingSpace` and not the fully qualified type `LivingSpace SquareFeet`.
+Notice below that `LivingSpace` is a function from Type to Type.  For example, if you give it type `SquareFeet`
+it gives you back the fully qualified type `LivingSpace SquareFeet`.
+
+-/
+#check LivingSpace -- Type → Type
+/-!
+
+So the `instance : Functor` then is operating on the more abstract, or generic `LivingSpace` saying
+for the whole family of types `LivingSpace α` you can map to `LivingSpace β` using the generic
+`LivingSpace.map` map function by simply providing a function that does the more primitive mapping
+from `(f : α → β)`.  So `LivingSpace.map` is a sort of function applicator.
+This is called a "higher order function" because it takes a function as input
+`(α → β)` and returns another function as output `F α → F β`.
+
+Notice that `LivingSpace.map` applies a function `f` to convert the units of all the LivingSpace
+fields, except for `numBedrooms` which is a count (and therefore is not a measurement that needs
+converting).
+
+So now you can define a simple conversion function, let's say you want square meters instead:
+
+-/
+abbrev SquareMeters := Float
+def squareFeetToMeters (ft : SquareFeet ) : SquareMeters := (ft / 10.7639104)
+/-!
+
+and now bringing it all together you can use the simple function `squareFeetToMeters` to map
+`mySpace` to square meters:
+
+-/
+#eval mySpace.map squareFeetToMeters
+/-
+{ totalSize := 167.225472,
+  numBedrooms := 4,
+  masterBedroomSize := 46.451520,
+  livingRoomSize := 83.612736,
+  kitchenSize := 37.161216 }
+  -/
+/-!
+
+Lean also defines custom infix operator `<$>` for `Functor.map` which allows you to write this:
+-/
+#eval (fun s => s.length) <$> ["elephant", "tiger", "giraffe"] -- [8, 5, 7]
+#eval (fun x => x + 1) <$> (some 5) -- some 6
+/-!
+
+Note that the infix operator is left associative which means it binds more tightly to the
+function on the left than to the expression on the right, this means you can often drop the
+parentheses on the right like this:
+
+-/
+#eval (fun x => x + 1) <$> some 5 -- some 6
+/-!
+
+Note that Lean lets you define your own syntax, so `<$>` is nothing special.
+You can define your own infix operator like this:
+
+-/
+infixr:100 " doodle " => Functor.map
+
+#eval (· * 5) doodle [1, 2, 3]  -- [5, 10, 15]
+
+/-!
+Wow, this is pretty powerful.  By providing a functor instance on `LivingSpace` with an
+implementation of the `map` function it is now super easy for anyone to come along and
+transform the units of a `LivingSpace` using very simple functions like `squareFeetToMeters`. Notice
+that squareFeetToMeters knows nothing about `LivingSpace`.
+
+## How do Functors help with Monads ?
+
+Functors are an abstract mathematical structure that is represented in Lean with a type class. The
+Lean functor defines both `map` and a special case for working on constants more efficiently called
+`mapConst`:
+
+```lean
+class Functor (f : Type u → Type v) : Type (max (u+1) v) where
+  map : {α β : Type u} → (α → β) → f α → f β
+  mapConst : {α β : Type u} → α → f β → f α
+```
+
+Note that `mapConst` has a default implementation, namely:
+`mapConst : {α β : Type u} → α → f β → f α := Function.comp map (Function.const _)` in the `Functor`
+type class.  So you can use this default implementation and you only need to replace it if
+your functor has a more specialized variant than this (usually the custom version is more performant).
+
+In general then, a functor is a function on types `F : Type u → Type v` equipped with an operator
+called `map` such that if you have a function `f` of type `α → β` then `map f` will convert your
+container type from `F α → F β`. This corresponds to the category-theory notion of
+[functor](https://en.wikipedia.org/wiki/Functor) in the special case where the category is the
+category of types and functions between them.
+
+Understanding abstract mathematical structures is a little tricky for most people. So it helps to
+start with a simpler idea like functors before you try to understand monads.  Building on
+functors is the next abstraction called [Applicatives](applicatives.lean.md).
+-/

doc/monads/intro.md

@@ -0,0 +1,63 @@
+# Monads
+
+Monads are used heavily in Lean, as they are also in Haskell. Monads come from the wonderful world
+of [Category Theory](https://en.wikipedia.org/wiki/Monad_%28category_theory%29).
+
+Monads in Lean are so similar to Haskell that this introduction to monads is heavily based on the
+similar chapter of the [Monday Morning Haskell](https://mmhaskell.com/monads/). Many thanks to
+the authors of that material for allowing us to reuse it here.
+
+Monads build on the following fundamental type classes which you will need to understand
+first before fully understanding monads. Shown in light blue are some concrete functors
+and monads that will also be covered in this chapter:
+
+![image](../images/monads.svg)
+
+This chapter is organized to give you a bottom up introduction to monads, starting with functors and
+applicative functors, you'll get an intuition for how these abstract structures work in Lean. Then
+you'll dive into monads and learn how to use some of the most useful built-in ones.
+
+## [Functor](functors.lean.md)
+A functor is a type class that provides a map function and the map function is something many
+people are already familiar with so this should be easy to follow.  Here you will see some
+concrete examples in action with `List` and `Option`.
+
+## [Applicative Functors](applicatives.lean.md)
+Applicatives are a little more difficult to understand than functors, but their functionality can
+still be summed up in a couple simple functions.  Here you will learn how to create an
+`Applicative List` and a completely custom `Applicative` type.
+
+## [Monads Tutorial](monads.lean.md)
+Now that you have an intuition for how abstract structures work, you'll examine some of the problems
+that functors and applicative functors don't help you solve. Then you'll learn the specifics of how
+to actually use monads with some examples using the `Option` monad and the all important `IO` monad.
+
+## [Reader Monad](readers.lean.md)
+Now that you understand the details of what makes a monadic structure work, in this section, you'll
+learn about one of the most useful built in monads `ReaderM`, which gives your programs a
+global read-only context.
+
+## [State Monad](states.lean.md)
+This section introduces the `StateM` monad. This monad allows you to access a particular type that you can
+both read from and write to. It opens the door to fully stateful programming, allowing you to do many
+of the things a function programming language supposedly "can't" do.
+
+## [Except Monad](except.lean.md)
+
+Similar to the `Option` monad the `Except` monad allows you to change the signature of a function so
+that it can return an `ok` value or an `error` and it provides the classic exception handling
+operations `throw/try/catch` so that your programs can do monad-based exception handling.
+
+## [Monad Transformers](transformers.lean.md)
+
+Now that you are familiar with all the above monads it is time to answer the question - how you can
+make them work together? After all, there are definitely times when you need multiple kinds of
+monadic behavior. This section introduces the concept of monad transformers, which allow you to
+combine multiple monads into one.
+
+## [Monad Laws](laws.lean.md)
+This section examines what makes a monad a legal monad. You could just implement your monadic type
+classes any way you want and write "monad" instances, but starting back with functors and
+applicative functors, you'll learn that all these structures have "laws" that they are expected to
+obey with respect to their behavior. You can make instances that don't follow these laws. But you do
+so at your peril, as other programmers will be very confused when they try to use them.

doc/monads/laws.lean

@@ -0,0 +1,322 @@
+/-!
+# Monad Laws
+
+In the previous sections you learned how to use [Functors](functors.lean.md),
+[Applicatives](applicatives.lean.md), and [Monads](monads.lean.md), and you played with some useful
+instances including [Option](monads.lean.md), [IO](monads.lean.md), [Reader](readers.lean.md),
+[State](states.lean.md) and [Except](except.lean.md) and you learned about composition using [Monad
+Transformers](transformers.lean.md).
+
+So far, you've learned the concrete details you need in order to _use_ monads in your Lean programs.
+But there's still one more important concept you need if you want to _create_ new functors,
+applicatives and monads. Namely, the notion of _structural "laws"_ -- rules that these type
+classes should follow in order to meet other programmers' expectations about your code.
+
+## Life without Laws
+
+Remember Lean represents each of these abstract structures by a type class. Each of these type classes
+has one or two main functions. So, as long as you implement those functions and it type checks, you
+have a new functor, applicative, or monad, right?
+
+Well not quite. Yes, your program will compile and you'll be able to use the instances. But this
+doesn't mean your instances follow the mathematical constructs. If they don't, your instances won't
+fulfill other programmers' expectations. Each type class has its own "laws". For instance, suppose
+you have the following Point Functor:
+-/
+structure Point (α : Type) where
+  x : α
+  y : α
+  deriving Repr, BEq
+
+def Point.map (f : α → β) (s : Point α) : Point β :=
+  { x := f s.y,  -- an example of something weird
+    y := f s.x }
+
+instance : Functor Point where
+  map := Point.map
+
+#eval (·+2) <$> (Point.mk 1 2) -- { x := 4, y := 3 }
+
+/-!
+This Point does something weird, when you `map` it because it transposes the `x` and `y` coordinates
+which is not what other people would expect from a `map` function.  In fact, it breaks the rules
+as you will see below.
+
+## What are the Functor laws?
+
+Functors have two laws: the _identity_ law, and the _composition_ law. These laws express behaviors that
+your functor instances should follow. If they don't, other programmers will be very confused at the
+effect your instances have on their program.
+
+The identity law says that if you "map" the identity function (`id`) over your functor, the
+resulting functor should be the same. A succinct way of showing this on a `List` functor is:
+
+-/
+def list1 := [1,2,3]
+
+#eval id <$> list1 == list1 -- true
+/-!
+
+Now let's try the same test on the `Point` functor:
+-/
+
+def p1 : Point Nat := (Point.mk 1 2)
+
+#eval id <$> p1 == p1 -- false
+
+/-!
+Oh, and look while the `List` is behaving well, the `Point` functor fails this identity test.
+
+The _composition_ law says that if you "map" two functions in succession over a functor, this
+should be the same as "composing" the functions and simply mapping that one super-function over the
+functor.  In Lean you can compose two functions using `Function.comp f g` (or the syntax `f ∘ g`,
+which you can type in VS code using `\o `) and you will get the same results from both of these
+showing that the composition law holds for `List Nat`:
+
+-/
+def double (x : Nat) := x + x
+def square (x : Nat) := x * x
+
+#eval double <$> (square <$> list1) -- [2, 8, 18]
+
+#eval (double <$> (square <$> list1)) == ((double ∘ square) <$> list1) -- true
+
+-- ok, what about the Point class?
+#eval double <$> (square <$> p1) -- { x := 2, y := 8 }
+#eval (double ∘ square) <$> p1   -- { x := 8, y := 2 }
+
+#eval double <$> (square <$> p1) == (double ∘ square) <$> p1  -- false
+/-!
+Note that composition also fails on the bad `Point` because the x/y transpose.
+
+As you can see this bad `Point` implementation violates both of the functor laws. In this case it
+would not be a true functor. Its behavior would confuse any other programmers trying to use it. You
+should take care to make sure that your instances make sense. Once you get a feel for these type
+classes, the likelihood is that the instances you'll create will follow the laws.
+
+You can also write a bad functor that passes one law but not the other like this:
+-/
+def bad_option_map {α β : Type u} : (α → β) → Option α → Option β
+  | _, _ => none
+
+instance : Functor Option where
+    map := bad_option_map
+
+def t1 : Option Nat := some 10
+
+#eval id <$> t1 == t1 -- false
+#eval double <$> (square <$> t1) == (double ∘ square) <$> t1  -- true
+/-!
+
+This fails the id law but obeys the composition law. Hopefully this explains the value of these
+laws, and you don't need to see any more bad examples!
+
+## What are the Applicative Laws?
+
+While functors have two laws, applicatives have four laws:
+
+- Identity
+- Homomorphism
+- Interchange
+- Composition
+
+### Identity
+
+`pure id <*> v = v`
+
+Applying the identity function through an applicative structure should not change the underlying
+values or structure.  For example:
+-/
+instance : Applicative List where
+  pure := List.pure
+  seq f x := List.bind f fun y => Functor.map y (x ())
+
+#eval pure id <*> [1, 2, 3]  -- [1, 2, 3]
+/-!
+
+The `pure id` statement here is wrapping the identity function in an applicative structure
+so that you can apply that over the container `[1, 2, 3]` using the Applicative `seq` operation
+which has the notation `<*>`.
+
+To prove this for all values `v` and any applicative `m` you can write this theorem:
+-/
+example [Applicative m] [LawfulApplicative m] (v : m α) :
+  pure id <*> v = v :=
+  by simp -- Goals accomplished 🎉
+/-!
+
+### Homomorphism
+
+`pure f <*> pure x = pure (f x)`
+
+Suppose you wrap a function and an object in `pure`. You can then apply the wrapped function over the
+wrapped object. Of course, you could also apply the normal function over the normal object, and then
+wrap it in `pure`. The homomorphism law states these results should be the same.
+
+For example:
+
+-/
+def x := 1
+def f := (· + 2)
+
+#eval pure f <*> pure x = (pure (f x) : List Nat) -- true
+/-!
+
+You should see a distinct pattern here. The overriding theme of almost all these laws is that these
+`Applicative` types should behave like normal containers. The `Applicative` functions should not
+have any side effects. All they should do is facilitate the wrapping, unwrapping, and transformation
+of data contained in the container resulting in a new container that has the same structure.
+
+### Interchange
+
+`u <*> pure y = pure (. y) <*> u`.
+
+This law is is a little more complicated, so don't sweat it too much. It states that the order that
+you wrap things shouldn't matter. One the left, you apply any applicative `u` over a pure wrapped
+object. On the right, you first wrap a function applying the object as an argument. Note that `(·
+y)` is short hand for: `fun f => f y`. Then you apply this to the first applicative `u`. These
+should be the same.
+
+For example:
+
+-/
+def y := 4
+def g : List (Nat → Nat) := [(· + 2)]
+
+#eval g <*> pure y = pure (· y) <*> g      -- true
+/-!
+
+You can prove this with the following theorem:
+-/
+example [Applicative m] [LawfulApplicative m] (u : m (α → β)) (y : α) :
+  u <*> pure y = pure (· y) <*> u :=
+  by simp [pure_seq] -- Goals accomplished 🎉
+
+/-!
+
+### Composition:
+
+`u <*> v <*> w = u <*> (v <*> w)`
+
+This final applicative law mimics the second functor law. It is a composition law. It states that
+function composition holds across applications within the applicative:
+
+For example:
+
+-/
+def u := [1, 2]
+def v := [3, 4]
+def w := [5, 6]
+
+#eval pure (·+·+·) <*> u <*> v <*> w
+-- [9, 10, 10, 11, 10, 11, 11, 12]
+
+#eval let grouping := pure (·+·) <*> v <*> w
+      pure (·+·) <*> u <*> grouping
+-- [9, 10, 10, 11, 10, 11, 11, 12]
+/-!
+
+To test composition you see the separate grouping `(v <*> w)` then that can be used in the outer
+sequence `u <*> grouping` to get the same final result `[9, 10, 10, 11, 10, 11, 11, 12]`.
+
+## What are the Monad Laws?
+
+Monads have three laws:
+
+- Left Identity
+- Right Identity
+- Associativity
+
+### Left Identity
+
+Identity laws for monads specify that `pure` by itself shouldn't really change anything about the
+structure or its values.
+
+Left identity is `x >>= pure = x` and is demonstrated by the following examples on a monadic `List`:
+-/
+instance : Monad List  where
+  pure := List.pure
+  bind := List.bind
+
+def a := ["apple", "orange"]
+
+#eval a >>= pure      -- ["apple", "orange"]
+
+#eval a >>= pure = a  -- true
+
+/-!
+
+### Right Identity
+
+Right identity is `pure x >>= f = f x` and is demonstrated by the following example:
+-/
+def h (x : Nat) : Option Nat :=  some (x + 1)
+def z := 5
+
+#eval pure z >>= h                  -- some 6
+#eval h z                           -- some 6
+
+#eval pure z >>= h = h z            -- true
+/-!
+
+So in this example, with this specific `z` and `h`, you see that the rule holds true.
+
+
+### Associativity
+
+The associativity law is written as:
+```lean,ignore
+  x >>= f >>= g = x >>= (λ x => f x >>= g)
+```
+where `(x : m α)` and `(f : α → m β)` and `(g : β → m γ)`.
+
+The associativity law is difficult to parse like some of the applicative laws, but what it is saying
+is that if you change the grouping of `bind` operations, you should still get the same result.
+This law has a parallel structure to the other composition laws.
+
+You can see this in action in the following rewrite of `runOptionFuncsBind` from [monads](monads.lean.md):
+-/
+def optionFunc1 : String -> Option Nat
+  | "" => none
+  | str => some str.length
+
+def optionFunc2 (i : Nat) : Option Float :=
+  if i % 2 == 0 then none else some (i.toFloat * 3.14159)
+
+def optionFunc3 (f : Float) : Option (List Nat) :=
+  if f > 15.0 then none else some [f.floor.toUInt32.toNat, f.ceil.toUInt32.toNat]
+
+
+def runOptionFuncsBind (input : String) : Option (List Nat) :=
+   optionFunc1 input >>= optionFunc2 >>= optionFunc3
+
+def runOptionFuncsBindGrouped (input : String) : Option (List Nat) :=
+   optionFunc1 input >>= (λ x => optionFunc2 x >>= optionFunc3)
+
+#eval runOptionFuncsBind "big"        -- some [9, 10]
+#eval runOptionFuncsBindGrouped "big" -- some [9, 10]
+/-!
+
+Notice here we had to insert a `λ` function just like the definition says: `(λ x => f x >>= g)`.
+This is because unlike applicatives, you can't resolve the structure of later operations without the
+results of earlier operations quite as well because of the extra context monads provide. But you can
+still group their later operations into composite functions taking their inputs from earlier on, and
+the result should be the same.
+
+
+## Summary
+
+While these laws may be a bit difficult to understand just by looking at them, the good news is that
+most of the instances you'll make will naturally follow the laws so long as you keep it simple, so
+you shouldn't have to worry about them too much.
+
+There are two main ideas from all the laws:
+
+1. Applying the identity or pure function should not change the underlying values or structure.
+1. It should not matter what order you group operations in.  Another way to state this is function
+   composition should hold across your structures.
+
+Following these laws will ensure other programmers are not confused by the behavior of your
+new functors, applicatives and monads.
+
+-/

doc/monads/monads.lean

@@ -0,0 +1,300 @@
+/-!
+# Monads
+
+Building on [Functors](functors.lean.md) and [Applicatives](applicatives.lean.md) we can now
+introduce [monads](https://en.wikipedia.org/wiki/Monad_%28category_theory%29).
+
+A monad is another type of abstract, functional structure. Let's explore what makes it different
+from the first two structures.
+
+## What is a Monad?
+
+A monad is a computational context. It provides a structure that allows you to chain together
+operations that have some kind of shared state or similar effect. Whereas pure functional code can
+only operate on explicit input parameters and affect the program through explicit return values,
+operations in a monad can affect other computations in the chain implicitly through side effects,
+especially modification of an implicitly shared value.
+
+## How are monads represented in Lean?
+
+Like functors and applicatives, monads are represented with a type class in Lean:
+
+```lean,ignore
+class Monad (m : Type u → Type v) extends Applicative m, Bind m where
+```
+
+Just as every applicative is a functor, every monad is also an applicative and there's one more new
+base type class used here that you need to understand, namely, `Bind`.
+
+```lean,ignore
+class Bind (f : Type u → Type v) where
+  bind : {α β : Type u} → f α → (α → f β) → f β
+```
+
+The `bind` operator also has infix notation `>>=` where `x >>= g` represents the result of executing
+`x` to get a value of type `f α` then unwrapping the value `α` from that and passing it to function
+`g` of type `α → f β` returning the result of type `f β` where `f` is the target structure type
+(like `Option` or List)
+
+This `bind` operation looks similar to the other ones you've seen so far, if you put them all
+together `Monad` has the following operations:
+
+```lean,ignore
+class Monad (f : Type u → Type v) extends Applicative f, Bind f where
+  pure {α : Type u} : α → f α
+  map : {α β : Type u} → (α → β) → f α → f β
+  seq : {α β : Type u} → f (α → β) → (Unit → f α) → f β
+  bind : {α β : Type u} → f α → (α → f β) → f β
+  ...
+```
+
+Notice `Monad` also contains `pure` it must also have a "default" way to wrap a value in the
+structure.
+
+The `bind` operator is similar to the applicative `seq` operator in that it chains two operations,
+with one of them being function related. Notice that `bind`, `seq` and `map` all take a function of
+some kind.  Let's examine those function types:
+
+- map: `(α → β)`
+- seq: `f (α → β)`
+- bind: `(α → f β)`
+
+So `map` is a pure function, `seq` is a pure function wrapped in the structure, and `bind` takes a
+pure input but produces an output wrapped in the structure.
+
+Note: we are ignoring the `(Unit → f α)` function used by `seq` here since that has a special
+purpose explained in [Applicatives Lazy Evaluation](applicatives.lean.md#lazy-evaluation).
+
+## Basic Monad Example
+
+Just as `Option` is a functor and an applicative functor, it is also a monad! Let's start with how
+`Option` implements the Monad type class.
+
+-/
+instance : Monad Option where
+  pure := Option.some
+  bind := Option.bind
+/-!
+
+where:
+
+```lean,ignore
+def Option.bind : Option α → (α → Option β) → Option β
+  | none,   _ => none
+  | some a, f => f a
+```
+
+> **Side note**: this function definition is using a special shorthand syntax in Lean where the `:=
+match a, b with` code can be collapsed away. To make this more clear consider the following simpler
+example, where `Option.bind` is using the second form like `bar`:
+
+-/
+def foo (x : Option Nat) (y : Nat) : Option Nat :=
+  match x, y with
+  | none, _ => none
+  | some x, y => some (x + y)
+
+def bar : Option Nat → Nat → Option Nat
+  | none, _ => none
+  | some x, y => some (x + y)
+
+#eval foo (some 1) 2  -- some 3
+#eval bar (some 1) 2  -- some 3
+/-!
+What is important is that `Option.bind` is using a `match` statement to unwrap the input value
+`Option α`, if it is `none` then it does nothing and returns `none`, if it has a value of type `α`
+then it applies the function in the second argument `(α → Option β)` to this value, which is
+the expression `f a` that you see in the line `  | some a, f => f a` above.  The function
+returns a result of type `Option β` which then becomes the return value for `bind`.  So there
+is no structure wrapping required on the return value since the input function already did that.
+
+But let's bring in the definition of a monad. What does it mean to describe `Option` as a
+computational context?
+
+The `Option` monad encapsulates the context of failure. Essentially, the `Option` monad lets us
+abort a series of operations whenever one of them fails. This allows future operations to assume
+that all previous operations have succeeded. Here's some code to motivate this idea:
+
+-/
+def optionFunc1 : String -> Option Nat
+  | "" => none
+  | str => some str.length
+
+def optionFunc2 (i : Nat) : Option Float :=
+  if i % 2 == 0 then none else some (i.toFloat * 3.14159)
+
+def optionFunc3 (f : Float) : Option (List Nat) :=
+  if f > 15.0 then none else some [f.floor.toUInt32.toNat, f.ceil.toUInt32.toNat]
+
+def runOptionFuncs (input : String) : Option (List Nat) :=
+  match optionFunc1 input with
+  | none => none
+  | some i => match optionFunc2 i with
+    | none => none
+    | some f => optionFunc3 f
+
+#eval runOptionFuncs "big" -- some [9, 10]
+/-!
+
+Here you see three different functions that could fail. These are then combined in `runOptionFuncs`.
+But then you have to use nested `match` expressions to check if the previous result succeeded. It
+would be very tedious to continue this pattern much longer.
+
+The `Option` monad helps you fix this. Here's what this function looks like using the `bind`
+operator.
+
+-/
+
+def runOptionFuncsBind (input : String) : Option (List Nat) :=
+  optionFunc1 input >>= optionFunc2 >>= optionFunc3
+
+#eval runOptionFuncsBind "big" -- some [9, 10]
+/-!
+
+It's much cleaner now! You take the first result and pass it into the second and third functions
+using the `bind` operation. The monad instance handles all the failure cases so you don't have to!
+
+Let's see why the types work out. The result of `optionFunc1` input is simply `Option Nat`. Then the
+bind operator allows you to take this `Option Nat` value and combine it with `optionFunc2`, whose type
+is `Nat → Option Float` The **bind operator resolves** these to an `Option Float`. Then you pass this
+similarly through the bind operator to `optionFunc3`, resulting in the final type, `Option (List Nat)`.
+
+Your functions will not always combine so cleanly though. This is where `do` notation comes into play.
+This notation allows you to write monadic operations one after another, line-by-line. It almost makes
+your code look like imperative programming. You can rewrite the above as:
+-/
+
+def runOptionFuncsDo (input : String) : Option (List Nat) := do
+  let i ← optionFunc1 input
+  let f ← optionFunc2 i
+  optionFunc3 f
+
+#eval runOptionFuncsDo "big" -- some [9, 10]
+/-!
+
+The `←` operator used here is special. It effectively unwraps the value on the right-hand side from
+the monad. This means the value `i` has type `Nat`, _even though_ the result of `optionFunc1` is
+`Option Nat`. This is done using a `bind` operation under the hood.
+
+> Note you can use `<-` or the nice unicode symbol `←` which you can type into VS code by typing
+these characters `\l `.  When you type the final space, `\l` is replaced with `←`.
+
+Observe that we do not unwrap the final line of the computation. The function result is `Option
+(List Nat)` which matches what `optionFunc3` returns. At first glance, this may look more complicated
+than the `bind` example. However, it gives you a lot more flexibility, like mixing monadic and
+non-monadic statements, using if then/else structures with their own local do blocks and so on. It
+is particularly helpful when one monadic function depends on multiple previous functions.
+
+## Example using List
+
+You can easily make `List` into a monad with the following, since List already provides an
+implementation of `pure` and `bind`.
+
+-/
+instance : Monad List  where
+  pure := List.pure
+  bind := List.bind
+/-!
+
+Like you saw with the applicative `seq` operator, the `bind` operator applies the given function
+to every element of the list.  It is useful to look at the bind implementation for List:
+
+-/
+open List
+def bind (a : List α) (b : α → List β) : List β := join (map b a)
+/-!
+
+So `Functor.map` is used to apply the function `b` to every element of `a` but this would
+return a whole bunch of little lists, so `join` is used to turn those back into a single list.
+
+Here's an example where you use `bind` to convert a list of strings into a combined list of chars:
+
+-/
+
+#eval "apple".toList  -- ['a', 'p', 'p', 'l', 'e']
+
+#eval ["apple", "orange"] >>= String.toList
+-- ['a', 'p', 'p', 'l', 'e', 'o', 'r', 'a', 'n', 'g', 'e']
+
+/-!
+
+
+## The IO Monad
+
+The `IO Monad` is perhaps the most important monad in Lean. It is also one of the hardest monads to
+understand starting out. Its actual implementation is too intricate to discuss when first learning
+monads. So it is best to learn by example.
+
+What is the **computational context** that describes the IO monad? IO operations can read
+information from or write information to the terminal, file system, operating system, and/or
+network. They interact with systems outside of your program. If you want to get user input, print a
+message to the user, read information from a file, or make a network call, you'll need to do so
+within the IO Monad.
+
+The state of the world outside your program can change at virtually any moment, and so this IO
+context is particularly special. So these IO operations are "side effects" which means you cannot
+perform them from "pure" Lean functions.
+
+Now, the most important job of pretty much any computer program is precisely to perform this
+interaction with the outside world. For this reason, the root of all executable Lean code is a
+function called main, with the type `IO Unit`. So every program starts in the IO monad!
+
+When your function is `IO` monadic, you can get any input you need, call into "pure" code with the
+inputs, and then output the result in some way. The reverse does not work. You cannot call into IO
+code from pure code like you can call into a function that takes `Option` as input. Another way to
+say this is you cannot invent an `IO` context out of thin air, it has to be given to you in your
+`main` function.
+
+Let's look at a simple program showing a few of the basic IO functions. It also uses `do` notation
+to make the code read nicely:
+-/
+def main : IO Unit := do
+  IO.println "enter a line of text:"
+  let stdin ← IO.getStdin            -- IO IO.FS.Stream (monadic)
+  let input ← stdin.getLine          -- IO.FS.Stream → IO String (monadic)
+  let uppercased := input.toUpper    -- String → String (pure)
+  IO.println uppercased              -- IO Unit (monadic)
+/-!
+
+So, once again you can see that the `do` notation lets you chain a series of monadic actions.
+`IO.getStdin` is of type `IO IO.FS.Stream` and `stdin.getLine` is of type `IO String`
+and `IO.println` is of type `IO Unit`.
+
+In between you see a non-monadic expression `let uppercased := input.toUpper` which is fine too.
+A let statement can occur in any monad. Just as you could unwrap `i` from `Option Nat` to get the
+inner Nat, you can use `←` to unwrap the result of `getLine` to get a String. You can then manipulate
+this value using normal pure string functions like `toUpper`, and then you can pass the result to the
+`IO.println` function.
+
+This is a simple echo program. It reads a line from the terminal, and then prints the line back out
+capitalized to the terminal. Hopefully it gives you a basic understanding of how IO works.
+
+You can test this program using `lean --run` as follows:
+
+```
+> lean --run Main.lean
+enter a line of text:
+the quick brown fox
+THE QUICK BROWN FOX
+```
+
+Here the user entered the string `the quick brown fox` and got back the uppercase result.
+
+## What separates Monads from Applicatives?
+
+The key that separates these is **context**. You cannot really determine the structure of
+"future" operations without knowing the results of "past" operations, because the past can alter the
+context in which the future operations work. With applicatives, you can't get the final function
+result without evaluating everything, but you can determine the structure of how the operation will
+take place. This allows some degree of parallelism with applicatives that is not generally possible
+with monads.
+
+
+## Conclusion
+
+Hopefully you now have a basic level understanding of what a monad is. But perhaps some more
+examples of what a "computational context" means would be useful to you. The Reader, State and
+Except monads each provide a concrete and easily understood context that can be compared easily to
+function parameters. You can learn more about those in [Reader monads](readers.lean.md),
+[State monads](states.lean.md), and the [Except monad](except.lean.md).
+-/