enhance: test on Windows and fix what it found (#1348)

Adds a weekly and on-demand workflow that runs the cross-compiled tests on Windows Server 2025 Core in a [dockur/windows](https://github.com/dockur/windows) QEMU VM, and fixes the Windows bugs it found in host and UNC paths, env handling, shell quoting, post-task scripts, health checks, checkout, action symlinks, step summaries, Git client certificates and docker mode from a Windows host.

Host-mode steps now emit UTF-8, `bash` falls back to Git's bash instead of the WSL launcher, the runner can run as a native Windows service, steps can keep background helpers with `RUNNER_TRACKING_ID`, and Windows arm64 binaries are released.

Fixes: https://gitea.com/gitea/runner/issues/1339
Fixes: https://gitea.com/gitea/runner/issues/637
Fixes: https://gitea.com/gitea/runner/issues/1172
Fixes: https://gitea.com/gitea/runner/issues/347
Reviewed-on: https://gitea.com/gitea/runner/pulls/1348
Reviewed-by: bircni <bircni@icloud.com>
This commit is contained in:
silverwind committed 2026-11-03 01:26:10 +00:00
1 parent 6e42621b4a
commit d6f5b68a0d
85 files changed
+2578 -620

No files matched your search

+21
View File
@@ -0,0 +1,21 @@
name: windows
on:
schedule:
- cron: '0 3 * * 1'
workflow_dispatch:
env:
DOCKER_CONFIG: /tmp/docker-noauth # see test.yml
jobs:
test:
name: test on windows
runs-on: ubuntu-latest
timeout-minutes: 180
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- uses: ./.gitea/actions/go-setup
- name: prepare anonymous docker config
run: mkdir -p "$DOCKER_CONFIG" && echo '{}' > "$DOCKER_CONFIG/config.json"
- name: test on windows
run: make test-windows
-2
View File
@@ -34,8 +34,6 @@ builds:
goarch: s390x goarch: s390x
- goos: windows - goos: windows
goarch: arm goarch: arm
- goos: windows
goarch: arm64
- goos: freebsd - goos: freebsd
goarch: s390x goarch: s390x
- goos: freebsd - goos: freebsd
+9
View File
@@ -69,3 +69,12 @@ Docker and Kubernetes.
The suite shares one Gitea, regular runner and k3s cluster. Cache and ephemeral scenarios use isolated The suite shares one Gitea, regular runner and k3s cluster. Cache and ephemeral scenarios use isolated
repository runners. A run path is `<workflow>@<ref>` and a log row is `<timestamp>Z <payload>`. repository runners. A run path is `<workflow>@<ref>` and a log row is `<timestamp>Z <payload>`.
## Windows tests
`make test-windows` cross-compiles the test binaries, installs Windows Server 2025 Core in a
[dockur/windows](https://github.com/dockur/windows) QEMU VM on the docker host and runs them there.
The host must be amd64 and allow privileged containers. `WINDOWS_TEST_TIMEOUT` sets the minutes
allowed for installing Windows and running the tests, 165 by default. The VM uses KVM when the host
has `/dev/kvm`, otherwise Windows is emulated and much slower. Docker-backed tests skip in the VM
and the race detector is off, it needs cgo. CI runs this weekly and on demand.
+5
View File
@@ -8,6 +8,7 @@ E2E_JOB_IMAGE ?= docker.gitea.com/runner-images:ubuntu-latest@sha256:657378e5cfd
SERVICE_IMAGE ?= nginx:1.31.7-alpine@sha256:892bb2152464f0c5d8972593a5f5afaf8054552580b699d95f8ff4af08c2d33d # renovate: datasource=docker SERVICE_IMAGE ?= nginx:1.31.7-alpine@sha256:892bb2152464f0c5d8972593a5f5afaf8054552580b699d95f8ff4af08c2d33d # renovate: datasource=docker
E2E_GITEA_IMAGE ?= gitea/gitea:main-nightly E2E_GITEA_IMAGE ?= gitea/gitea:main-nightly
E2E_K3S_IMAGE ?= rancher/k3s:v1.37.1-k3s1@sha256:ca7f37d993d82ef0dcdcfecb2e0e2618ea541dbaffc620c8cedebe01a82acd0d # renovate: datasource=docker E2E_K3S_IMAGE ?= rancher/k3s:v1.37.1-k3s1@sha256:ca7f37d993d82ef0dcdcfecb2e0e2618ea541dbaffc620c8cedebe01a82acd0d # renovate: datasource=docker
WINDOWS_TEST_IMAGE ?= dockurr/windows:6.06@sha256:16fd7c39a3e48bb946d29c59dc6015942e5b900a1fa5d7e5552053bfc48ee2ea # renovate: datasource=docker
GOFILES := $(shell find . -type f -name "*.go" -o -name "go.mod" ! -name "generated.*") GOFILES := $(shell find . -type f -name "*.go" -o -name "go.mod" ! -name "generated.*")
@@ -150,6 +151,10 @@ coverage-report: ## turn coverage.txt from `make test` into .tmp/coverage.md
test-dind: ## run the daemon-facing tests against the built dind image or podman (TARGET=dind|dind-rootless|podman) test-dind: ## run the daemon-facing tests against the built dind image or podman (TARGET=dind|dind-rootless|podman)
@PODMAN_TEST_IMAGE=$(PODMAN_TEST_IMAGE) ./scripts/test-dind.sh $(TARGET) @PODMAN_TEST_IMAGE=$(PODMAN_TEST_IMAGE) ./scripts/test-dind.sh $(TARGET)
.PHONY: test-windows
test-windows: ## run the tests on Windows Server Core in a QEMU VM on the docker host
@WINDOWS_TEST_IMAGE=$(WINDOWS_TEST_IMAGE) ./scripts/test-windows.sh
E2E_CONCURRENCY ?= 8 E2E_CONCURRENCY ?= 8
.PHONY: test-e2e .PHONY: test-e2e
+22 -1
View File
@@ -2,6 +2,8 @@
## Installation ## Installation
The runner runs on Linux, macOS and Windows.
### Prerequisites ### Prerequisites
On the runner: On the runner:
@@ -89,6 +91,23 @@ If the registry succeed, it will run immediately. Next time, you could run the r
./gitea-runner daemon ./gitea-runner daemon
``` ```
### Windows service
Register the runner first, keeping `gitea-runner.exe`, `config.yaml` and the `.runner` registration file in the same directory. From an elevated Command Prompt, install and start the service with absolute paths:
```bat
sc.exe create gitea-runner binPath= "\"C:\gitea-runner\gitea-runner.exe\" daemon --config \"C:\gitea-runner\config.yaml\"" start= auto
sc.exe start gitea-runner
```
The service runs from the executable's directory and appends its logs to `gitea-runner.log` there. Its account needs write access to that directory and access to the tools used by jobs. Use `sc.exe stop gitea-runner` to stop it, allowing running jobs to finish according to `runner.shutdown_timeout`.
On shutdown or reboot, Windows waits for the service's preshutdown timeout at most, so set it above `runner.shutdown_timeout`, here six minutes for `5m`. It takes effect after the next reboot:
```bat
reg.exe add HKLM\SYSTEM\CurrentControlSet\Services\gitea-runner /v PreshutdownTimeout /t REG_DWORD /d 360000 /f
```
### Run with docker ### Run with docker
The recommended Docker-in-Docker image runs its own Docker daemon for jobs. Jobs never get the host's Docker socket, which is equivalent to root access on the host. The recommended Docker-in-Docker image runs its own Docker daemon for jobs. Jobs never get the host's Docker socket, which is equivalent to root access on the host.
@@ -233,7 +252,9 @@ The `daemon` command also accepts `--labels` (which defaults to the `GITEA_RUNNE
Whenever the resulting labels differ from the ones in the registration file, they are written back to it and re-declared to the Gitea instance on startup. Whenever the resulting labels differ from the ones in the registration file, they are written back to it and re-declared to the Gitea instance on startup.
> **Note:** A runner that only exposes `host` labels still needs access to a Docker daemon (e.g. a mounted `/var/run/docker.sock`) whenever a job uses a `docker://` action or a service container. `host` labels only change where the job's own steps run; container-based steps and actions are still executed with Docker. > **Note:** A runner that only exposes `host` labels still needs access to a Docker daemon (e.g. a mounted `/var/run/docker.sock`) whenever a job uses a `docker://` action or a service container. `host` labels only change where the job's own steps run; container-based steps and actions are still executed with Docker. Windows hosts do not support container actions and `docker://` steps in host jobs, like GitHub's Windows runners.
On Windows hosts, a step's background processes end with the job unless it opts out, see [host-mode background processes](docs/host-mode.md).
#### Service containers #### Service containers
+1 -1
View File
@@ -281,7 +281,7 @@ func TestSafeResolve(t *testing.T) {
for name, tc := range tests { for name, tc := range tests {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
require.Equal(t, tc.want, safeResolve(baseDir, tc.input)) require.Equal(t, filepath.FromSlash(tc.want), safeResolve(baseDir, tc.input))
}) })
} }
} }
+163 -3
View File
@@ -10,6 +10,7 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt"
"io" "io"
"math/rand/v2" "math/rand/v2"
"net" "net"
@@ -29,6 +30,10 @@ import (
var actionFetches sync.Map var actionFetches sync.Map
var symlink = (*os.Root).Symlink
var symlinkFallback = runtime.GOOS == "windows"
const treeSizes = ".sizes" // beside the tree rather than in it, which is copied into job containers as it is const treeSizes = ".sizes" // beside the tree rather than in it, which is copied into job containers as it is
type treeFile struct { type treeFile struct {
@@ -229,6 +234,8 @@ func extractTree(ctx context.Context, archive io.Reader, dir string, strip bool)
sha := "" sha := ""
sizes := map[string]int64{} sizes := map[string]int64{}
links := map[string]string{}
var deferred []string
reader := tar.NewReader(archive) reader := tar.NewReader(archive)
for { for {
header, err := reader.Next() header, err := reader.Next()
@@ -257,9 +264,8 @@ func extractTree(ctx context.Context, archive io.Reader, dir string, strip bool)
case tar.TypeReg: case tar.TypeReg:
sizes[name], err = writeTreeFile(root, name, header.Mode, reader) sizes[name], err = writeTreeFile(root, name, header.Mode, reader)
case tar.TypeSymlink: case tar.TypeSymlink:
if err = root.Symlink(header.Linkname, name); err != nil && runtime.GOOS == "windows" { links[filepath.Clean(name)] = header.Linkname
sizes[name], err = writeTreeFile(root, name, 0, strings.NewReader(header.Linkname)) deferred = append(deferred, filepath.Clean(name))
}
} }
if err != nil { if err != nil {
return "", err return "", err
@@ -268,12 +274,166 @@ func extractTree(ctx context.Context, archive io.Reader, dir string, strip bool)
if !isFullObjectID(sha) { if !isFullObjectID(sha) {
return "", errors.New("the archive names no commit") return "", errors.New("the archive names no commit")
} }
linksByParent := map[string][]string{}
if symlinkFallback {
for name := range links {
linksByParent[filepath.Dir(name)] = append(linksByParent[filepath.Dir(name)], filepath.Base(name))
}
}
for requireTarget := symlinkFallback; len(deferred) > 0; {
unresolved := deferred[:0]
for _, name := range deferred {
if err := ctx.Err(); err != nil {
return "", err
}
if requireTarget {
if _, err := root.Stat(filepath.Dir(name) + string(filepath.Separator) + links[name]); err != nil {
unresolved = append(unresolved, name)
continue
}
}
if err := symlink(root, links[name], name); err == nil || !symlinkFallback {
if err != nil {
return "", err
}
continue
}
if err := copyTreeLink(ctx, root, name, links, linksByParent, sizes); err != nil {
return "", fmt.Errorf("materialize symlink %q: %w", filepath.ToSlash(name), err)
}
}
if len(unresolved) == len(deferred) {
requireTarget = false
}
deferred = unresolved
}
if err := root.Close(); err != nil { if err := root.Close(); err != nil {
return "", err return "", err
} }
return sha, installTree(ctx, dir, sha, temp, sizes) return sha, installTree(ctx, dir, sha, temp, sizes)
} }
func resolveTreeLink(name string, links map[string]string) (string, bool) {
if !filepath.IsLocal(name) {
return "", false
}
parts := strings.Split(name, string(filepath.Separator))
var resolved []string
for count := 0; len(parts) > 0; {
part := parts[0]
parts = parts[1:]
switch part {
case "", ".":
continue
case "..":
if len(resolved) == 0 {
return "", false
}
resolved = resolved[:len(resolved)-1]
continue
}
target, ok := links[filepath.Join(append(resolved, part)...)]
if !ok {
resolved = append(resolved, part)
continue
}
count++
if target == "" || os.IsPathSeparator(target[0]) || filepath.VolumeName(target) != "" || count > 255 {
return "", false
}
parts = append(strings.Split(filepath.FromSlash(target), string(filepath.Separator)), parts...)
}
return filepath.Join(append([]string{"."}, resolved...)...), true
}
var (
errTreeLinkTarget = errors.New("unmaterializable symlink target")
errTreeLinkCycle = fmt.Errorf("cyclic %w", errTreeLinkTarget)
)
func copyTreeLink(ctx context.Context, root *os.Root, name string, links map[string]string, linksByParent map[string][]string, sizes map[string]int64) error {
writeLinkText := func(destination, target string) error { // as git writes links it cannot create
if err := root.RemoveAll(destination); err != nil {
return err
}
for written := range sizes {
if written == destination || strings.HasPrefix(written, destination+string(filepath.Separator)) {
delete(sizes, written)
}
}
size, err := writeTreeFile(root, destination, 0o644, strings.NewReader(target))
sizes[destination] = size
return err
}
active := map[string]bool{}
var copyTarget func(string, string) error
copyTarget = func(source, destination string) error {
if err := ctx.Err(); err != nil {
return err
}
source, ok := resolveTreeLink(source, links)
if !ok {
return errTreeLinkTarget
}
file, err := root.Open(source)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return errTreeLinkTarget
}
return err
}
defer file.Close()
info, err := file.Stat()
if err != nil {
return err
}
if info.Mode().IsRegular() {
sizes[destination], err = writeTreeFile(root, destination, int64(info.Mode().Perm()), file)
return err
}
if !info.IsDir() {
return errTreeLinkTarget
}
if active[source] {
return errTreeLinkCycle
}
active[source] = true
defer delete(active, source)
entries, err := file.ReadDir(-1)
if err != nil {
return err
}
children := map[string]bool{}
for _, entry := range entries {
children[entry.Name()] = true
}
for _, child := range linksByParent[source] {
children[child] = true
}
if err := root.MkdirAll(destination, info.Mode().Perm()); err != nil {
return err
}
for child := range children {
childSource, childDestination := filepath.Join(source, child), filepath.Join(destination, child)
err := copyTarget(childSource, childDestination)
if target, isLink := links[childSource]; isLink && errors.Is(err, errTreeLinkTarget) && !errors.Is(err, errTreeLinkCycle) {
err = writeLinkText(childDestination, target)
}
if err != nil {
return err
}
}
return nil
}
if err := copyTarget(name, name); !errors.Is(err, errTreeLinkTarget) {
return err
}
if err := ctx.Err(); err != nil {
return err
}
return writeLinkText(name, links[name])
}
var lockTree = AcquireCloneLock // replaced in tests to see installTree wait for a job holding the tree var lockTree = AcquireCloneLock // replaced in tests to see installTree wait for a job holding the tree
// installTree moves temp into place as dir/sha, replacing a tree that failed treeIntact. // installTree moves temp into place as dir/sha, replacing a tree that failed treeIntact.
+151 -1
View File
@@ -8,6 +8,7 @@ import (
"bytes" "bytes"
"context" "context"
"encoding/base64" "encoding/base64"
"errors"
"fmt" "fmt"
"net/http" "net/http"
"net/http/cgi" "net/http/cgi"
@@ -40,6 +41,7 @@ func TestFetchAction(t *testing.T) {
require.NoError(t, os.WriteFile(filepath.Join(remote, "run.sh"), []byte("#!/bin/sh\n"), 0o755)) require.NoError(t, os.WriteFile(filepath.Join(remote, "run.sh"), []byte("#!/bin/sh\n"), 0o755))
require.NoError(t, os.Symlink("action.yml", filepath.Join(remote, "link"))) require.NoError(t, os.Symlink("action.yml", filepath.Join(remote, "link")))
require.NoError(t, gitCmd("-C", remote, "add", ".")) require.NoError(t, gitCmd("-C", remote, "add", "."))
require.NoError(t, gitCmd("-C", remote, "update-index", "--chmod=+x", "run.sh"))
require.NoError(t, gitCmd("-C", remote, "commit", "-m", "initial")) require.NoError(t, gitCmd("-C", remote, "commit", "-m", "initial"))
require.NoError(t, gitCmd("-C", remote, "tag", "v1")) require.NoError(t, gitCmd("-C", remote, "tag", "v1"))
expectedSha := gitRevParse(t, remote, "main") expectedSha := gitRevParse(t, remote, "main")
@@ -192,7 +194,9 @@ func TestFetchAction(t *testing.T) {
assert.NoDirExists(t, filepath.Join(tree, ".git")) assert.NoDirExists(t, filepath.Join(tree, ".git"))
info, err := os.Stat(filepath.Join(tree, "run.sh")) info, err := os.Stat(filepath.Join(tree, "run.sh"))
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, os.FileMode(0o755), info.Mode().Perm()) if runtime.GOOS != "windows" {
assert.EqualValues(t, 0o755, info.Mode().Perm())
}
link, err := os.Readlink(filepath.Join(tree, "link")) link, err := os.Readlink(filepath.Join(tree, "link"))
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, "action.yml", link) assert.Equal(t, "action.yml", link)
@@ -391,6 +395,79 @@ func TestExtractTreeConfinesEntries(t *testing.T) {
} }
} }
func TestExtractTreeCopiesDeferredLinks(t *testing.T) {
originalSymlink, originalFallback := symlink, symlinkFallback
symlink = func(_root *os.Root, _target, _name string) error { return errors.New("symlink privilege unavailable") }
symlinkFallback = true
t.Cleanup(func() { symlink, symlinkFallback = originalSymlink, originalFallback })
for _, test := range []struct {
name string
links [][2]string
files []string
text bool
}{
{name: "file chain", links: [][2]string{{"index.js", "entry.js"}, {"entry.js", "dist/index.js"}}, files: []string{"index.js", "entry.js"}},
{name: "directory chain", links: [][2]string{{"lib", "bundle"}, {"bundle", "dist"}, {"dist/entry.js", "index.js"}}, files: []string{"lib/index.js", "lib/entry.js", "lib/nested/index.js", "bundle/index.js", "bundle/entry.js", "bundle/nested/index.js"}},
{name: "directory in target path", links: [][2]string{{"index.js", "bundle/nested/index.js"}, {"bundle", "dist"}}, files: []string{"index.js", "bundle/nested/index.js"}},
{name: "parent after directory link", links: [][2]string{{"index.js", "bundle/../index.js"}, {"bundle", "dist/nested"}}, files: []string{"index.js", "bundle/index.js"}},
{name: "outside", links: [][2]string{{"index.js", "../outside.js"}}, text: true},
{name: "absolute", links: [][2]string{{"index.js", t.TempDir()}}, text: true},
{name: "dangling", links: [][2]string{{"index.js", "missing.js"}}, text: true},
{name: "dangling child of directory link", links: [][2]string{{"lib", "dist"}, {"dist/optional", "missing.js"}}, files: []string{"lib/index.js", "lib/nested/index.js"}},
{name: "cycle", links: [][2]string{{"index.js", "entry.js"}, {"entry.js", "index.js"}}, text: true},
{name: "directory cycle", links: [][2]string{{"dist/loop", "."}}, text: true},
} {
t.Run(test.name, func(t *testing.T) {
sha := strings.Repeat("a", 40)
var archive bytes.Buffer
writer := tar.NewWriter(&archive)
require.NoError(t, writer.WriteHeader(&tar.Header{Typeflag: tar.TypeXGlobalHeader, PAXRecords: map[string]string{"comment": sha}}))
require.NoError(t, writer.WriteHeader(&tar.Header{Name: "dist", Typeflag: tar.TypeDir, Mode: 0o755}))
require.NoError(t, writer.WriteHeader(&tar.Header{Name: "dist/nested", Typeflag: tar.TypeDir, Mode: 0o755}))
for _, link := range test.links {
require.NoError(t, writer.WriteHeader(&tar.Header{Name: link[0], Typeflag: tar.TypeSymlink, Linkname: link[1]}))
}
content := "console.log('action');\n"
for _, name := range []string{"dist/index.js", "dist/nested/index.js"} {
require.NoError(t, writer.WriteHeader(&tar.Header{Name: name, Typeflag: tar.TypeReg, Mode: 0o755, Size: int64(len(content))}))
_, err := writer.Write([]byte(content))
require.NoError(t, err)
}
require.NoError(t, writer.Close())
dir := t.TempDir()
_, err := extractTree(t.Context(), &archive, dir, false)
require.NoError(t, err)
assert.True(t, treeIntact(dir, sha))
if test.text {
for _, link := range test.links {
data, err := os.ReadFile(filepath.Join(dir, sha, link[0]))
require.NoError(t, err)
assert.Equal(t, link[1], string(data))
info, err := os.Lstat(filepath.Join(dir, sha, link[0]))
require.NoError(t, err)
assert.True(t, info.Mode().IsRegular())
}
}
for _, name := range test.files {
data, err := os.ReadFile(filepath.Join(dir, sha, name))
require.NoError(t, err)
assert.Equal(t, content, string(data))
info, err := os.Lstat(filepath.Join(dir, sha, name))
require.NoError(t, err)
assert.True(t, info.Mode().IsRegular())
if runtime.GOOS != "windows" {
assert.Equal(t, os.FileMode(0o755), info.Mode().Perm())
}
require.NoError(t, os.WriteFile(filepath.Join(dir, sha, name), nil, 0o644))
assert.False(t, treeIntact(dir, sha), name)
require.NoError(t, os.WriteFile(filepath.Join(dir, sha, name), []byte(content), 0o755))
}
})
}
}
func TestTreeIntactFindsNamesThatAreNotUTF8(t *testing.T) { func TestTreeIntactFindsNamesThatAreNotUTF8(t *testing.T) {
if runtime.GOOS != "linux" { if runtime.GOOS != "linux" {
t.Skip("macOS and Windows store no name that is not UTF-8") t.Skip("macOS and Windows store no name that is not UTF-8")
@@ -410,3 +487,76 @@ func TestTreeIntactFindsNamesThatAreNotUTF8(t *testing.T) {
require.FileExists(t, filepath.Join(dir, sha, "caf\xe9.js")) require.FileExists(t, filepath.Join(dir, sha, "caf\xe9.js"))
assert.True(t, treeIntact(dir, sha)) assert.True(t, treeIntact(dir, sha))
} }
func TestExtractTreeCreatesLinksInPlatformOrder(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("requires symlink privileges")
}
original, originalFallback := symlink, symlinkFallback
var created []string
symlink = func(root *os.Root, target, name string) error {
created = append(created, name)
return root.Symlink(target, name)
}
t.Cleanup(func() { symlink, symlinkFallback = original, originalFallback })
var archive bytes.Buffer
writer := tar.NewWriter(&archive)
require.NoError(t, writer.WriteHeader(&tar.Header{Typeflag: tar.TypeXGlobalHeader, PAXRecords: map[string]string{"comment": strings.Repeat("a", 40)}}))
for _, entry := range []*tar.Header{
{Name: "a", Typeflag: tar.TypeSymlink, Linkname: "b"},
{Name: "b", Typeflag: tar.TypeSymlink, Linkname: "dist"},
{Name: "dangling", Typeflag: tar.TypeSymlink, Linkname: "missing"},
{Name: "dist", Typeflag: tar.TypeDir, Mode: 0o755},
} {
require.NoError(t, writer.WriteHeader(entry))
}
require.NoError(t, writer.Close())
for _, test := range []struct {
name string
fallback bool
want []string
}{
{name: "native", want: []string{"a", "b", "dangling"}},
{name: "fallback", fallback: true, want: []string{"b", "a", "dangling"}},
} {
t.Run(test.name, func(t *testing.T) {
symlinkFallback = test.fallback
created = nil
_, err := extractTree(t.Context(), bytes.NewReader(archive.Bytes()), t.TempDir(), false)
require.NoError(t, err)
assert.Equal(t, test.want, created)
})
}
}
func TestCopyTreeLinkCancellationBetweenEntries(t *testing.T) {
root, err := os.OpenRoot(t.TempDir())
require.NoError(t, err)
defer root.Close()
require.NoError(t, root.Mkdir("source", 0o755))
for _, name := range []string{"first", "second"} {
_, err := writeTreeFile(root, filepath.Join("source", name), 0o644, strings.NewReader("content"))
require.NoError(t, err)
}
sizes := map[string]int64{}
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
observed := &treeCopyContext{Context: ctx, check: func() {
if len(sizes) == 1 {
cancel()
}
}}
require.ErrorIs(t, copyTreeLink(observed, root, "copy", map[string]string{"copy": "source"}, nil, sizes), context.Canceled)
assert.Len(t, sizes, 1)
}
type treeCopyContext struct {
context.Context
check func()
}
func (ctx *treeCopyContext) Err() error {
ctx.check()
return ctx.Context.Err()
}
+32 -8
View File
@@ -5,6 +5,7 @@
package git package git
import ( import (
"bytes"
"context" "context"
"crypto/sha256" "crypto/sha256"
"encoding/base64" "encoding/base64"
@@ -27,6 +28,7 @@ import (
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging" "gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/internal/pkg/lock" "gitea.com/gitea/runner/internal/pkg/lock"
"gitea.com/gitea/runner/internal/pkg/process"
) )
var ( var (
@@ -387,6 +389,17 @@ func RemoteConfig(token string, insecureSkipTLS bool) []string {
return config return config
} }
func ClientCertConfig(prefix, certFile, keyFile, executionOS string) []string {
if certFile == "" {
return nil
}
config := []string{prefix + "sslCert=" + certFile, prefix + "sslKey=" + keyFile}
if strings.EqualFold(executionOS, "windows") {
config = append(config, prefix+"sslBackend=openssl")
}
return config
}
// ConfigEnv applies `-c` style entries through the environment, keeping secrets out of argv. // ConfigEnv applies `-c` style entries through the environment, keeping secrets out of argv.
func ConfigEnv(config ...string) map[string]string { func ConfigEnv(config ...string) map[string]string {
env := map[string]string{"GIT_CONFIG_COUNT": strconv.Itoa(len(config))} env := map[string]string{"GIT_CONFIG_COUNT": strconv.Itoa(len(config))}
@@ -444,9 +457,7 @@ func runGitWithStdin(ctx context.Context, dir string, input *NewGitCloneExecutor
config := []string{"core.autocrlf=false", "core.eol=lf"} // keep action files LF config := []string{"core.autocrlf=false", "core.eol=lf"} // keep action files LF
if input != nil { if input != nil {
config = append(config, RemoteConfig(input.Token, input.InsecureSkipTLS)...) config = append(config, RemoteConfig(input.Token, input.InsecureSkipTLS)...)
if input.ClientCertFile != "" { config = append(config, ClientCertConfig("http.", input.ClientCertFile, input.ClientKeyFile, runtime.GOOS)...)
config = append(config, "http.sslCert="+input.ClientCertFile, "http.sslKey="+input.ClientKeyFile)
}
} }
if dir != "" { if dir != "" {
args = append([]string{"-C", dir}, args...) args = append([]string{"-C", dir}, args...)
@@ -461,15 +472,28 @@ func runGitWithStdin(ctx context.Context, dir string, input *NewGitCloneExecutor
cmd.Env = append(cmd.Env, key+"="+value) cmd.Env = append(cmd.Env, key+"="+value)
} }
cmd.Env = append(cmd.Env, "GIT_TERMINAL_PROMPT=0") cmd.Env = append(cmd.Env, "GIT_TERMINAL_PROMPT=0")
if runtime.GOOS != "windows" { var treeKill *process.TreeKill
if runtime.GOOS == "windows" {
treeKill = process.NewTreeKill(cmd) // a cancelled git leaves its remote helper holding the repository otherwise
} else {
cmd.Cancel = func() error { return cmd.Process.Signal(os.Interrupt) } // lets git remove its lock files cmd.Cancel = func() error { return cmd.Process.Signal(os.Interrupt) } // lets git remove its lock files
} }
cmd.WaitDelay = time.Second cmd.WaitDelay = time.Second
var stdout bytes.Buffer
var stderr strings.Builder var stderr strings.Builder
cmd.Stdout = &stdout
cmd.Stderr = &stderr cmd.Stderr = &stderr
output, err := cmd.Output() err := cmd.Start()
if err != nil { if err == nil {
return output, fmt.Errorf("git %s: %w: %s", strings.Join(args, " "), err, strings.TrimSpace(stderr.String())) if treeKill != nil {
if killer, captureErr := treeKill.Capture(cmd.Process); captureErr == nil {
defer killer.Close()
}
}
err = cmd.Wait()
} }
return output, nil if err != nil {
return stdout.Bytes(), fmt.Errorf("git %s: %w: %s", strings.Join(args, " "), err, strings.TrimSpace(stderr.String()))
}
return stdout.Bytes(), nil
} }
+21 -4
View File
@@ -54,6 +54,22 @@ func TestMain(m *testing.M) {
os.Exit(code) os.Exit(code)
} }
func TestClientCertConfig(t *testing.T) {
for _, testcase := range []struct {
prefix, certFile, executionOS string
expected []string
}{
{"http.", "cert.pem", "windows", []string{"http.sslCert=cert.pem", "http.sslKey=key.pem", "http.sslBackend=openssl"}},
{"http.https://example.com/.", "cert.pem", "Windows", []string{"http.https://example.com/.sslCert=cert.pem", "http.https://example.com/.sslKey=key.pem", "http.https://example.com/.sslBackend=openssl"}},
{"http.", "cert.pem", "Linux", []string{"http.sslCert=cert.pem", "http.sslKey=key.pem"}},
{"http.", "", "Windows", nil},
} {
t.Run(testcase.prefix+testcase.certFile+testcase.executionOS, func(t *testing.T) {
assert.Equal(t, testcase.expected, ClientCertConfig(testcase.prefix, testcase.certFile, "key.pem", testcase.executionOS))
})
}
}
func TestFindGitSlug(t *testing.T) { func TestFindGitSlug(t *testing.T) {
for _, testcase := range []struct { for _, testcase := range []struct {
remoteURL string remoteURL string
@@ -185,7 +201,8 @@ func TestGitFindRef(t *testing.T) {
func TestGitCloneExecutor(t *testing.T) { func TestGitCloneExecutor(t *testing.T) {
remoteDir, workDir := newGitRemote(t) remoteDir, workDir := newGitRemote(t)
require.NoError(t, os.WriteFile(filepath.Join(workDir, "action.sh"), []byte("#!/bin/sh\necho hi\n"), 0o755)) require.NoError(t, os.WriteFile(filepath.Join(workDir, "action.sh"), []byte("#!/bin/sh\necho hi\n"), 0o755))
require.NoError(t, gitCmd("-C", workDir, "add", "action.sh")) require.NoError(t, os.WriteFile(filepath.Join(workDir, ".gitattributes"), []byte("*.sh text\n"), 0o644))
require.NoError(t, gitCmd("-C", workDir, "add", "action.sh", ".gitattributes"))
require.NoError(t, gitCmd("-C", workDir, "commit", "-m", "initial")) require.NoError(t, gitCmd("-C", workDir, "commit", "-m", "initial"))
require.NoError(t, gitCmd("-C", workDir, "tag", "v2")) require.NoError(t, gitCmd("-C", workDir, "tag", "v2"))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main")) require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main"))
@@ -200,7 +217,7 @@ func TestGitCloneExecutor(t *testing.T) {
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:refs/heads/"+fullSha[:4])) require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:refs/heads/"+fullSha[:4]))
hostConfig := filepath.Join(t.TempDir(), "gitconfig") hostConfig := filepath.Join(t.TempDir(), "gitconfig")
require.NoError(t, os.WriteFile(hostConfig, []byte("[core]\n\tautocrlf = true\n[url \""+remoteDir+"\"]\n\tinsteadOf = https://example.invalid/action\n"), 0o644)) require.NoError(t, os.WriteFile(hostConfig, []byte("[core]\n\tautocrlf = true\n\teol = crlf\n[url "+strconv.Quote(remoteDir)+"]\n\tinsteadOf = https://example.invalid/action\n"), 0o644))
t.Setenv("GIT_CONFIG_GLOBAL", hostConfig) t.Setenv("GIT_CONFIG_GLOBAL", hostConfig)
t.Setenv("GIT_DIR", t.TempDir()) t.Setenv("GIT_DIR", t.TempDir())
@@ -676,8 +693,8 @@ func TestNewGitCloneExecutorFetchSendsTokenAndHonoursContext(t *testing.T) {
select { select {
case <-reached: case <-reached:
case <-time.After(10 * time.Second): case err := <-done:
t.Fatal("the executor never reached the remote") t.Fatalf("the executor finished before reaching the remote: %v", err)
} }
assert.Equal(t, "Basic "+base64.StdEncoding.EncodeToString([]byte("token:secret")), authorization) assert.Equal(t, "Basic "+base64.StdEncoding.EncodeToString([]byte("token:secret")), authorization)
cancel() cancel()
+20
View File
@@ -0,0 +1,20 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package common
import "strings"
// LookupFold finds name in values, ignoring case where the environment does, as GitHub's inputs always do.
func LookupFold[V any](values map[string]V, name string, caseInsensitive bool) (V, bool) {
if value, ok := values[name]; ok || !caseInsensitive {
return value, ok
}
for key, value := range values {
if strings.EqualFold(key, name) {
return value, true
}
}
var zero V
return zero, false
}
+3
View File
@@ -10,6 +10,7 @@ import (
"fmt" "fmt"
"io" "io"
"path" "path"
"runtime"
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
@@ -20,6 +21,8 @@ import (
"github.com/moby/moby/api/types/container" "github.com/moby/moby/api/types/container"
) )
var hostOS = runtime.GOOS // replaced in tests
// ExitCodeError reports a non-zero process exit code from a container command. // ExitCodeError reports a non-zero process exit code from a container command.
type ExitCodeError int type ExitCodeError int
+6 -3
View File
@@ -125,6 +125,10 @@ func createBuildContext(ctx context.Context, contextDir, relDockerfile string) (
} }
func writeBuildContext(ctx context.Context, w io.Writer, contextDir string, includes []string, pm *patternmatcher.PatternMatcher) error { func writeBuildContext(ctx context.Context, w io.Writer, contextDir string, includes []string, pm *patternmatcher.PatternMatcher) error {
contextDir, err := filepath.EvalSymlinks(contextDir)
if err != nil {
return err
}
tw := tar.NewWriter(w) tw := tar.NewWriter(w)
seen := map[string]bool{} seen := map[string]bool{}
hardlinks := map[uint64]string{} hardlinks := map[uint64]string{}
@@ -132,8 +136,7 @@ func writeBuildContext(ctx context.Context, w io.Writer, contextDir string, incl
for _, include := range includes { for _, include := range includes {
include = filepath.FromSlash(include) include = filepath.FromSlash(include)
var parents []parentMatch var parents []parentMatch
root := strings.TrimSuffix(contextDir, string(filepath.Separator)) + string(filepath.Separator) + include // unlike filepath.Join, follows a symlinked context dir as docker build does err := filepath.WalkDir(filepath.Join(contextDir, include), func(file string, entry fs.DirEntry, err error) error {
err := filepath.WalkDir(root, func(file string, entry fs.DirEntry, err error) error {
if err != nil { if err != nil {
return leaveOut(ctx, file, err) return leaveOut(ctx, file, err)
} }
@@ -210,7 +213,7 @@ func addBuildContextEntry(ctx context.Context, tw *tar.Writer, file, rel string,
return leaveOut(ctx, file, err) return leaveOut(ctx, file, err)
} }
} }
header, err := tar.FileInfoHeader(noNameLookups{info}, link) header, err := tar.FileInfoHeader(noNameLookups{info}, filepath.ToSlash(link))
if err != nil { if err != nil {
return err return err
} }
+1 -1
View File
@@ -50,7 +50,7 @@ func TestCreateBuildContextFollowsDockerignoreAndSymlinkedRootButKeepsDockerfile
contexts := []string{dir} contexts := []string{dir}
if link := filepath.Join(t.TempDir(), "link"); os.Symlink(dir, link) == nil { if link := filepath.Join(t.TempDir(), "link"); os.Symlink(dir, link) == nil {
contexts = append(contexts, link+string(filepath.Separator)) contexts = append(contexts, link, link+string(filepath.Separator))
} }
for _, contextDir := range contexts { for _, contextDir := range contexts {
+8 -4
View File
@@ -351,7 +351,7 @@ type containerConfig struct {
// parse parses the args for the specified command and generates a Config, // parse parses the args for the specified command and generates a Config,
// a HostConfig and returns them with the specified command. // a HostConfig and returns them with the specified command.
// If the specified args are not valid, it will return an error. // If the specified args are not valid, it will return an error.
func parse(flags *pflag.FlagSet, copts *containerOptions, serverOS string) (*containerConfig, error) { //nolint:gocyclo,unparam // verbatim copy from docker/cli func parse(flags *pflag.FlagSet, copts *containerOptions, serverOS string) (*containerConfig, error) { //nolint:gocyclo // verbatim copy from docker/cli
var ( var (
attachStdin = copts.attach.Get("stdin") attachStdin = copts.attach.Get("stdin")
attachStdout = copts.attach.Get("stdout") attachStdout = copts.attach.Get("stdout")
@@ -724,6 +724,10 @@ func parse(flags *pflag.FlagSet, copts *containerOptions, serverOS string) (*con
}, nil }, nil
} }
func isUserDefinedNetwork(mode container.NetworkMode) bool { // Docker's helper uses the host OS instead of the daemon's.
return !mode.IsDefault() && mode != network.NetworkBridge && mode != network.NetworkHost && !mode.IsNone() && !mode.IsContainer()
}
// parseNetworkOpts converts --network advanced options to endpoint-specs, and combines // parseNetworkOpts converts --network advanced options to endpoint-specs, and combines
// them with the old --network-alias and --links. If returns an error if conflicting options // them with the old --network-alias and --links. If returns an error if conflicting options
// are found. // are found.
@@ -752,7 +756,7 @@ func parseNetworkOpts(copts *containerOptions) (map[string]*network.EndpointSett
} }
for i, n := range copts.netMode.Value() { for i, n := range copts.netMode.Value() {
if container.NetworkMode(n.Target).IsUserDefined() { if isUserDefinedNetwork(container.NetworkMode(n.Target)) {
hasUserDefined = true hasUserDefined = true
} else { } else {
hasNonUserDefined = true hasNonUserDefined = true
@@ -814,7 +818,7 @@ func applyContainerOptions(n *opts.NetworkAttachmentOpts, copts *containerOption
} }
// For a user-defined network, "--link" is an endpoint option, it creates an alias. But, // For a user-defined network, "--link" is an endpoint option, it creates an alias. But,
// for the default bridge it defines a legacy-link. // for the default bridge it defines a legacy-link.
if container.NetworkMode(n.Target).IsUserDefined() && copts.links.Len() > 0 { if isUserDefinedNetwork(container.NetworkMode(n.Target)) && copts.links.Len() > 0 {
n.Links = slices.Clone(copts.links.GetSlice()) n.Links = slices.Clone(copts.links.GetSlice())
} }
if copts.ipv4Address != nil { if copts.ipv4Address != nil {
@@ -840,7 +844,7 @@ func parseNetworkAttachmentOpt(ep opts.NetworkAttachmentOpts) (*network.Endpoint
if strings.TrimSpace(ep.Target) == "" { if strings.TrimSpace(ep.Target) == "" {
return nil, errors.New("no name set for network") return nil, errors.New("no name set for network")
} }
if !container.NetworkMode(ep.Target).IsUserDefined() { if !isUserDefinedNetwork(container.NetworkMode(ep.Target)) {
if len(ep.Aliases) > 0 { if len(ep.Aliases) > 0 {
return nil, errors.New("network-scoped aliases are only supported for user-defined networks") return nil, errors.New("network-scoped aliases are only supported for user-defined networks")
} }
+7
View File
@@ -484,6 +484,10 @@ type probeClient struct {
containers []container.Summary containers []container.Summary
} }
func (c *probeClient) Info(_ctx context.Context, _opts mobyclient.InfoOptions) (mobyclient.SystemInfoResult, error) {
return mobyclient.SystemInfoResult{}, nil
}
func (c *probeClient) Close() error { func (c *probeClient) Close() error {
return nil return nil
} }
@@ -766,6 +770,9 @@ func TestDaemonSeesDir(t *testing.T) {
} }
func TestRunnerContainerWorkdir(t *testing.T) { func TestRunnerContainerWorkdir(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("a runner container is Linux")
}
workdir := t.TempDir() workdir := t.TempDir()
t.Chdir(workdir) t.Chdir(workdir)
volume := container.MountPoint{Type: mount.TypeVolume, Source: "/daemon/_data", Destination: filepath.Dir(workdir)} volume := container.MountPoint{Type: mount.TypeVolume, Source: "/daemon/_data", Destination: filepath.Dir(workdir)}
+38 -15
View File
@@ -366,6 +366,7 @@ type containerReference struct {
id string id string
input *NewContainerInput input *NewContainerInput
pullPolicy string pullPolicy string
daemonOS string
UID int UID int
GID int GID int
// attachDone is closed by the attach() streaming goroutine once it has // attachDone is closed by the attach() streaming goroutine once it has
@@ -375,13 +376,18 @@ type containerReference struct {
LinuxContainerEnvironmentExtensions LinuxContainerEnvironmentExtensions
} }
func GetDockerClient(ctx context.Context) (cli client.APIClient, err error) { func GetDockerClient(ctx context.Context) (client.APIClient, error) {
cli, _, err := newDockerClient(ctx)
return cli, err
}
func newDockerClient(ctx context.Context) (cli client.APIClient, daemonOS string, err error) {
dockerHost := os.Getenv("DOCKER_HOST") dockerHost := os.Getenv("DOCKER_HOST")
if strings.HasPrefix(dockerHost, "ssh://") { if strings.HasPrefix(dockerHost, "ssh://") {
var args []string var args []string
if args, err = sshArgs(dockerHost); err != nil { if args, err = sshArgs(dockerHost); err != nil {
return nil, err return nil, "", err
} }
cli, err = client.New( cli, err = client.New(
client.WithHost("http://docker.example.com"), // the dialer below ignores the address client.WithHost("http://docker.example.com"), // the dialer below ignores the address
@@ -391,16 +397,17 @@ func GetDockerClient(ctx context.Context) (cli client.APIClient, err error) {
cli, err = client.New(client.FromEnv) cli, err = client.New(client.FromEnv)
} }
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to connect to docker daemon: %w", err) return nil, "", fmt.Errorf("failed to connect to docker daemon: %w", err)
} }
// Best-effort API version negotiation, matching the old client.NegotiateAPIVersion // Best-effort API version negotiation, matching the old client.NegotiateAPIVersion
// behaviour. Ping failures here are non-fatal: the connection is exercised again on // behaviour. Ping failures here are non-fatal: the connection is exercised again on
// the first real call, and the client falls back to the daemon's default API version. // the first real call, and the client falls back to the daemon's default API version.
if _, err := cli.Ping(ctx, client.PingOptions{NegotiateAPIVersion: true}); err != nil { ping, err := cli.Ping(ctx, client.PingOptions{NegotiateAPIVersion: true})
if err != nil {
common.Logger(ctx).Warnf("docker daemon ping during version negotiation failed, continuing: %v", err) common.Logger(ctx).Warnf("docker daemon ping during version negotiation failed, continuing: %v", err)
} }
return cli, nil return cli, ping.OSType, nil
} }
func GetHostInfo(ctx context.Context) (system.Info, error) { func GetHostInfo(ctx context.Context) (system.Info, error) {
@@ -440,16 +447,32 @@ func RunnerArch(ctx context.Context) string {
return info.Architecture return info.Architecture
} }
func (cr *containerReference) targetOS(ctx context.Context) string {
if cr.daemonOS == "" {
cr.daemonOS = "linux"
if cr.cli != nil {
if result, err := cr.cli.Info(ctx, client.InfoOptions{}); err == nil && result.Info.OSType != "" {
cr.daemonOS = result.Info.OSType
}
}
}
return cr.daemonOS
}
func (cr *containerReference) markExecutable(ctx context.Context) bool {
return hostOS == "windows" && cr.targetOS(ctx) == "linux"
}
func (cr *containerReference) connect() common.Executor { func (cr *containerReference) connect() common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
if cr.cli != nil { if cr.cli != nil {
return nil return nil
} }
cli, err := GetDockerClient(ctx) cli, daemonOS, err := newDockerClient(ctx)
if err != nil { if err != nil {
return err return err
} }
cr.cli = cli cr.cli, cr.daemonOS = cli, daemonOS
return nil return nil
} }
} }
@@ -459,6 +482,7 @@ func (cr *containerReference) Close() common.Executor {
if cr.cli != nil { if cr.cli != nil {
err := cr.cli.Close() err := cr.cli.Close()
cr.cli = nil cr.cli = nil
cr.daemonOS = ""
if err != nil { if err != nil {
return fmt.Errorf("failed to close client: %w", err) return fmt.Errorf("failed to close client: %w", err)
} }
@@ -603,7 +627,8 @@ func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config
copts.privileged = false copts.privileged = false
} }
containerConfig, err := parse(flags, copts, runtime.GOOS) serverOS := cr.targetOS(ctx)
containerConfig, err := parse(flags, copts, serverOS)
if err != nil { if err != nil {
return fmt.Errorf("cannot process container options: '%s': '%w'", options, err) return fmt.Errorf("cannot process container options: '%s': '%w'", options, err)
} }
@@ -618,7 +643,7 @@ func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config
// For Gitea, forcing --privileged off is not enough, other options reach the host too // For Gitea, forcing --privileged off is not enough, other options reach the host too
if !hostConfig.Privileged { if !hostConfig.Privileged {
trusted, err := parseOptionsHostConfig(cr.input.RunnerOptions) trusted, err := parseOptionsHostConfig(cr.input.RunnerOptions, serverOS)
if err != nil { if err != nil {
return err return err
} }
@@ -731,9 +756,7 @@ func (cr *containerReference) create(capAdd, capDrop []string) common.Executor {
cr.sanitizeConfig(ctx, hostConfig) cr.sanitizeConfig(ctx, hostConfig)
var networkingConfig *network.NetworkingConfig var networkingConfig *network.NetworkingConfig
n := hostConfig.NetworkMode if isUserDefinedNetwork(hostConfig.NetworkMode) && len(input.NetworkAliases) > 0 {
// IsUserDefined and IsHost are broken on windows
if n.IsUserDefined() && n != "host" && len(input.NetworkAliases) > 0 {
networkingConfig = &network.NetworkingConfig{ networkingConfig = &network.NetworkingConfig{
EndpointsConfig: map[string]*network.EndpointSettings{ EndpointsConfig: map[string]*network.EndpointSettings{
input.NetworkMode: {Aliases: input.NetworkAliases}, input.NetworkMode: {Aliases: input.NetworkAliases},
@@ -919,7 +942,7 @@ func (cr *containerReference) copyDir(dstPath, srcPath string, useGitIgnore, ski
if actPath := cr.GetActPath(); strings.HasPrefix(dstPath+"/", actPath+"/") { if actPath := cr.GetActPath(); strings.HasPrefix(dstPath+"/", actPath+"/") {
extractionRoot = actPath // keeps tar entries from crossing the image's /var/run symlink extractionRoot = actPath // keeps tar entries from crossing the image's /var/run symlink
} }
if err := writeDirTar(ctx, tarFile, path.Join("/", strings.TrimPrefix(dstPath, extractionRoot)), srcPath, useGitIgnore, skipGitDir, cr.UID, cr.GID); err != nil { if err := writeDirTar(ctx, tarFile, path.Join("/", strings.TrimPrefix(dstPath, extractionRoot)), srcPath, useGitIgnore, skipGitDir, cr.UID, cr.GID, cr.markExecutable(ctx)); err != nil {
return err return err
} }
@@ -1090,12 +1113,12 @@ func resetOption[T any](logger *logging.Logger, option string, field *T, trusted
// parseOptionsHostConfig parses one options string on its own, to see what it alone asks for. // parseOptionsHostConfig parses one options string on its own, to see what it alone asks for.
// Even "" goes through the parser, or its empty slices and maps would differ from a real parse. // Even "" goes through the parser, or its empty slices and maps would differ from a real parse.
func parseOptionsHostConfig(options string) (*container.HostConfig, error) { func parseOptionsHostConfig(options, serverOS string) (*container.HostConfig, error) {
flags, copts, _, err := parseContainerOptions(options) flags, copts, _, err := parseContainerOptions(options)
if err != nil { if err != nil {
return nil, err return nil, err
} }
containerConfig, err := parse(flags, copts, runtime.GOOS) containerConfig, err := parse(flags, copts, serverOS)
if err != nil { if err != nil {
return nil, fmt.Errorf("cannot process container options: '%s': '%w'", options, err) return nil, fmt.Errorf("cannot process container options: '%s': '%w'", options, err)
} }
+135 -2
View File
@@ -27,10 +27,12 @@ import (
"gitea.com/gitea/runner/act/common/logging/logtest" "gitea.com/gitea/runner/act/common/logging/logtest"
cerrdefs "github.com/containerd/errdefs" cerrdefs "github.com/containerd/errdefs"
"github.com/kballard/go-shellquote"
"github.com/moby/moby/api/pkg/stdcopy" "github.com/moby/moby/api/pkg/stdcopy"
"github.com/moby/moby/api/types/container" "github.com/moby/moby/api/types/container"
"github.com/moby/moby/api/types/mount" "github.com/moby/moby/api/types/mount"
"github.com/moby/moby/api/types/network" "github.com/moby/moby/api/types/network"
"github.com/moby/moby/api/types/system"
mobyclient "github.com/moby/moby/client" mobyclient "github.com/moby/moby/client"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock" "github.com/stretchr/testify/mock"
@@ -65,6 +67,14 @@ func TestDockerEnvironmentUsesContainerPath(t *testing.T) {
type mockDockerClient struct { type mockDockerClient struct {
mobyclient.APIClient mobyclient.APIClient
mock.Mock mock.Mock
info system.Info
infoErr error
infoCalls int
}
func (m *mockDockerClient) Info(_ctx context.Context, _opts mobyclient.InfoOptions) (mobyclient.SystemInfoResult, error) {
m.infoCalls++
return mobyclient.SystemInfoResult{Info: m.info}, m.infoErr
} }
func (m *mockDockerClient) ServerVersion(ctx context.Context, opts mobyclient.ServerVersionOptions) (mobyclient.ServerVersionResult, error) { func (m *mockDockerClient) ServerVersion(ctx context.Context, opts mobyclient.ServerVersionOptions) (mobyclient.ServerVersionResult, error) {
@@ -689,11 +699,11 @@ func TestMergeContainerConfigsDropsVolumeDriversFromWorkflows(t *testing.T) {
} }
func TestMergeContainerConfigsKeepsTheRunnersFilesAndEnvToItself(t *testing.T) { func TestMergeContainerConfigsKeepsTheRunnersFilesAndEnvToItself(t *testing.T) {
hostFile := filepath.Join(t.TempDir(), "host.env") hostFile := filepath.Join(t.TempDir(), "host file.env")
require.NoError(t, os.WriteFile(hostFile, []byte("STOLEN=from-the-host\n"), 0o600)) require.NoError(t, os.WriteFile(hostFile, []byte("STOLEN=from-the-host\n"), 0o600))
t.Setenv("RUNNER_SECRET", "s3cr3t") t.Setenv("RUNNER_SECRET", "s3cr3t")
for _, option := range []string{"--env-file " + hostFile, "--label-file " + hostFile} { for _, option := range []string{shellquote.Join("--env-file", hostFile), shellquote.Join("--label-file", hostFile)} {
logger := logging.New(logtest.New()) logger := logging.New(logtest.New())
cr := &containerReference{input: &NewContainerInput{NetworkMode: "bridge", WorkflowOptions: option}} cr := &containerReference{input: &NewContainerInput{NetworkMode: "bridge", WorkflowOptions: option}}
@@ -933,6 +943,7 @@ func TestContainerInfoFromInspect(t *testing.T) {
"/etc/hosts": "", "/etc/hosts": "",
"/etc/hostname": "", "/etc/hostname": "",
}, info.Mounts) }, info.Mounts)
assert.Equal(t, "/var/lib/docker/volumes/job/_data/subdir", info.DaemonPath("/workspace/owner/repo/subdir"))
}) })
} }
@@ -998,6 +1009,39 @@ func TestDockerCreateKeepsInputEntrypoint(t *testing.T) {
} }
} }
func TestDockerCreateNetworkAliases(t *testing.T) {
for _, test := range []struct {
networkMode string
wantAliases bool
}{
{"job-network", true},
{"host", false},
{"bridge", false},
{"default", false},
{"none", false},
{"container:x", false},
} {
t.Run(test.networkMode, func(t *testing.T) {
cr := &containerReference{
input: &NewContainerInput{NetworkMode: test.networkMode, NetworkAliases: []string{"service"}},
cli: &probeClient{create: func(_ctx context.Context, opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
assert.Equal(t, container.NetworkMode(test.networkMode), opts.HostConfig.NetworkMode)
if test.wantAliases {
assert.Equal(t, &network.NetworkingConfig{EndpointsConfig: map[string]*network.EndpointSettings{
"job-network": {Aliases: []string{"service"}},
}}, opts.NetworkingConfig)
} else {
assert.Nil(t, opts.NetworkingConfig)
}
return mobyclient.ContainerCreateResult{ID: "created"}, nil
}},
}
require.NoError(t, cr.create(nil, nil)(t.Context()))
assert.Equal(t, "created", cr.id)
})
}
}
func TestMergeContainerConfigsKeepsNetworkAliasesFromOptions(t *testing.T) { func TestMergeContainerConfigsKeepsNetworkAliasesFromOptions(t *testing.T) {
logger := logging.New(logtest.New()) logger := logging.New(logtest.New())
cr := &containerReference{input: &NewContainerInput{ cr := &containerReference{input: &NewContainerInput{
@@ -1019,3 +1063,92 @@ func TestSupportsContainerImagePlatformDaemonError(t *testing.T) {
require.ErrorContains(t, err, "cannot connect to the Docker daemon") require.ErrorContains(t, err, "cannot connect to the Docker daemon")
assert.False(t, supported) assert.False(t, supported)
} }
func TestContainerOptionsUseDaemonOS(t *testing.T) {
for _, testcase := range []struct {
name string
serverOS string
infoErr error
device string
expected container.DeviceMapping
}{
{name: "Linux daemon", serverOS: "linux", device: "/dev/kvm:/dev/kvm:rwm", expected: container.DeviceMapping{PathOnHost: "/dev/kvm", PathInContainer: "/dev/kvm", CgroupPermissions: "rwm"}},
{name: "unknown daemon", device: "/dev/kvm:/dev/kvm:rwm", expected: container.DeviceMapping{PathOnHost: "/dev/kvm", PathInContainer: "/dev/kvm", CgroupPermissions: "rwm"}},
{name: "unavailable daemon info", infoErr: errors.New("unavailable"), device: "/dev/kvm:/dev/kvm:rwm", expected: container.DeviceMapping{PathOnHost: "/dev/kvm", PathInContainer: "/dev/kvm", CgroupPermissions: "rwm"}},
{name: "Windows daemon", serverOS: "windows", device: "class/01234567-89ab-cdef-0123-456789abcdef", expected: container.DeviceMapping{PathOnHost: "class/01234567-89ab-cdef-0123-456789abcdef"}},
} {
t.Run(testcase.name, func(t *testing.T) {
cli := &mockDockerClient{info: system.Info{OSType: testcase.serverOS}, infoErr: testcase.infoErr}
cr := &containerReference{cli: cli, input: &NewContainerInput{RunnerOptions: "--device=" + testcase.device, WorkflowOptions: "--device=" + testcase.device, NetworkMode: "bridge"}}
for range 2 {
hostConfig := &container.HostConfig{}
require.NoError(t, cr.mergeContainerConfigs(t.Context(), &container.Config{}, hostConfig))
assert.Equal(t, []container.DeviceMapping{testcase.expected}, hostConfig.Devices)
}
assert.Equal(t, 1, cli.infoCalls)
})
}
}
func TestConnectTakesDaemonOSFromPing(t *testing.T) {
var infos atomic.Int32
daemon := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
writer.Header().Set("API-Version", "1.47")
writer.Header().Set("OSType", "windows")
if strings.HasSuffix(request.URL.Path, "/info") {
infos.Add(1)
}
}))
t.Cleanup(daemon.Close)
t.Setenv("DOCKER_HOST", daemon.URL)
cr := &containerReference{input: &NewContainerInput{}}
require.NoError(t, cr.connect()(t.Context()))
assert.Equal(t, "windows", cr.targetOS(t.Context()))
assert.Zero(t, infos.Load())
}
func TestContainerCopyModes(t *testing.T) {
for _, testcase := range []struct {
hostOS, serverOS string
expected int64
}{
{"windows", "linux", 0o755},
{"windows", "windows", 0o666},
{"linux", "linux", 0o666},
} {
t.Run(testcase.hostOS+"-"+testcase.serverOS, func(t *testing.T) {
source := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(source, "run.sh"), []byte("#!/bin/sh\n"), 0o666))
require.NoError(t, os.Chmod(filepath.Join(source, "run.sh"), 0o666))
file := &FileEntry{Name: "run.sh", Mode: 0o600, Body: "#!/bin/sh\n"}
original := hostOS
hostOS = testcase.hostOS
t.Cleanup(func() { hostOS = original })
cli := &mockDockerClient{info: system.Info{OSType: testcase.serverOS}}
var modes []int64
cli.On("CopyToContainer", mock.Anything, "job", mock.Anything).
Run(func(args mock.Arguments) {
reader := tar.NewReader(args.Get(2).(mobyclient.CopyToContainerOptions).Content)
header, err := reader.Next()
require.NoError(t, err)
modes = append(modes, header.Mode)
assert.Equal(t, "run.sh", filepath.Base(header.Name))
content, err := io.ReadAll(reader)
require.NoError(t, err)
assert.Equal(t, file.Body, string(content))
_, err = reader.Next()
require.ErrorIs(t, err, io.EOF)
}).Return(mobyclient.CopyToContainerResult{}, nil).Twice()
cr := &containerReference{cli: cli, id: "job", input: &NewContainerInput{}}
require.NoError(t, cr.copyContent("/workspace", file)(t.Context()))
require.NoError(t, cr.copyDir("/workspace", source+string(filepath.Separator), false, true)(t.Context()))
assert.Equal(t, []int64{0o600, testcase.expected}, modes)
if testcase.hostOS == "windows" {
assert.Equal(t, 1, cli.infoCalls)
} else {
assert.Zero(t, cli.infoCalls)
}
cli.AssertExpectations(t)
})
}
}
+12 -1
View File
@@ -4,7 +4,18 @@
package container package container
import "context" import (
"context"
"path"
"strings"
)
func JoinPath(elem ...string) string { // path.Join would collapse the leading // of a UNC act path on a Windows host
if len(elem) > 0 && strings.HasPrefix(elem[0], "//") {
return "/" + path.Join(elem...)
}
return path.Join(elem...)
}
type ExecutionsEnvironment interface { type ExecutionsEnvironment interface {
Container Container
+20 -84
View File
@@ -40,7 +40,8 @@ type HostEnvironment struct {
// when Workdir points at a caller-owned checkout (e.g. `act` local mode). // when Workdir points at a caller-owned checkout (e.g. `act` local mode).
CleanWorkdir bool CleanWorkdir bool
ActPath string ActPath string
CleanUp func() Root string
TrackingID string // RUNNER_TRACKING_ID of the job, a step with another value keeps its leftovers
StdOut io.Writer StdOut io.Writer
AllocatePTY bool // allocate a pseudo-TTY for each step's process AllocatePTY bool // allocate a pseudo-TTY for each step's process
@@ -261,6 +262,7 @@ func (e *HostEnvironment) exec(ctx context.Context, command []string, cmdline st
// Flush any buffered, not-yet-newline-terminated trailing line, as the docker backend // Flush any buffered, not-yet-newline-terminated trailing line, as the docker backend
// does in waitForCommand, so the final line of a command's output is not lost. // does in waitForCommand, so the final line of a command's output is not lost.
defer common.FlushWriter(e.StdOut) defer common.FlushWriter(e.StdOut)
process.UTF8Console()
f, err := lookupPathHost(command[0], env, e.StdOut) f, err := lookupPathHost(command[0], env, e.StdOut)
if err != nil { if err != nil {
@@ -270,7 +272,12 @@ func (e *HostEnvironment) exec(ctx context.Context, command []string, cmdline st
cmd.Path = f cmd.Path = f
cmd.Args = command cmd.Args = command
cmd.Stdout = e.StdOut cmd.Stdout = e.StdOut
cmd.Env = getEnvListFromMap(env) trackingID := e.TrackingID
if value, ok := common.LookupFold(env, "RUNNER_TRACKING_ID", e.IsEnvironmentCaseInsensitive()); ok {
trackingID = value
}
tracked := trackingID == e.TrackingID
cmd.Env = append(getEnvListFromMap(env), "RUNNER_TRACKING_ID="+trackingID)
cmd.Stderr = e.StdOut cmd.Stderr = e.StdOut
cmd.Dir = wd cmd.Dir = wd
cmd.SysProcAttr = process.SysProcAttr(cmdline, false) cmd.SysProcAttr = process.SysProcAttr(cmdline, false)
@@ -278,6 +285,7 @@ func (e *HostEnvironment) exec(ctx context.Context, command []string, cmdline st
// Kills the step's whole tree on cancellation and bounds the post-exit I/O // Kills the step's whole tree on cancellation and bounds the post-exit I/O
// wait, so an orphan holding cmd's stdout pipe cannot hang cmd.Wait(). // wait, so an orphan holding cmd's stdout pipe cannot hang cmd.Wait().
treeKill := process.NewTreeKill(cmd) treeKill := process.NewTreeKill(cmd)
treeKill.KeepLeftovers = !tracked
var ppty *os.File var ppty *os.File
var tty *os.File var tty *os.File
@@ -311,10 +319,10 @@ func (e *HostEnvironment) exec(ctx context.Context, command []string, cmdline st
if err := cmd.Start(); err != nil { if err := cmd.Start(); err != nil {
return err return err
} }
// Assign before the step's Killer so the step's job nests inside the group's; if tracked {
// cancellation still scopes to this step's tree. if err := e.processGroup(ctx).Assign(cmd.Process); err != nil {
if err := e.processGroup(ctx).Assign(cmd.Process); err != nil { common.Logger(ctx).Warnf("could not assign the step's process to the job's process group, a process it leaves behind may outlive the job: %v", err)
common.Logger(ctx).Warnf("could not assign the step's process to the job's process group; a process it leaves behind may outlive the job: %v", err) }
} }
if k, kerr := treeKill.Capture(cmd.Process); kerr != nil { if k, kerr := treeKill.Capture(cmd.Process); kerr != nil {
common.Logger(ctx).Warnf("process tree kill setup failed, falling back to single-process kill: %v", kerr) common.Logger(ctx).Warnf("process tree kill setup failed, falling back to single-process kill: %v", kerr)
@@ -365,9 +373,7 @@ func (e *HostEnvironment) UpdateFromEnv(srcPath string, env *map[string]string)
// removeAll is a var so tests can substitute a blocking stub. // removeAll is a var so tests can substitute a blocking stub.
var removeAll = os.RemoveAll var removeAll = os.RemoveAll
// removeAllWithContext returns once the delete finishes or ctx is cancelled. On // Returns on cancel while an uninterruptible delete keeps running, freeing the runner's capacity slot.
// cancellation the goroutine leaks: a delete inside a syscall cannot be
// interrupted (see runWithTimeout).
func removeAllWithContext(ctx context.Context, path string) error { func removeAllWithContext(ctx context.Context, path string) error {
done := make(chan error, 1) done := make(chan error, 1)
go func() { done <- removeAll(path) }() go func() { done <- removeAll(path) }()
@@ -409,35 +415,6 @@ func removePathWithRetry(ctx context.Context, path string) error {
return lastErr return lastErr
} }
// buildWindowsWorkspaceKillScript builds a PowerShell command that taskkills
// every process tree whose ExecutablePath or CommandLine references one of the
// given workspace dirs, releasing file handles for cleanup. Win32_Process
// exposes both fields (Get-Process doesn't, wmic is deprecated); matching is on
// the dir+separator prefix via ordinal String methods, so a name-prefix sibling
// (job1 vs job10) is spared and path metacharacters stay literal.
func buildWindowsWorkspaceKillScript(dirs []string) string {
quoted := make([]string, len(dirs))
for i, d := range dirs {
// Single-quoted PowerShell literal; escape ' by doubling it.
quoted[i] = "'" + strings.ReplaceAll(d, "'", "''") + "'"
}
return `$paths = @(` + strings.Join(quoted, ",") + `)
$selfPid = $PID
Get-CimInstance Win32_Process -ErrorAction SilentlyContinue | Where-Object {
if ($_.ProcessId -eq $selfPid) { return $false }
foreach ($p in $paths) {
$prefix = $p + '\'
if ($_.ExecutablePath -and $_.ExecutablePath.StartsWith($prefix, [System.StringComparison]::OrdinalIgnoreCase)) { return $true }
if ($_.CommandLine -and $_.CommandLine.IndexOf($prefix, [System.StringComparison]::OrdinalIgnoreCase) -ge 0) { return $true }
}
return $false
} | ForEach-Object {
& taskkill.exe /PID $_.ProcessId /T /F 2>$null | Out-Null
}
`
}
func (e *HostEnvironment) terminateRunningProcesses(ctx context.Context) { func (e *HostEnvironment) terminateRunningProcesses(ctx context.Context) {
if runtime.GOOS != "windows" { if runtime.GOOS != "windows" {
return return
@@ -471,20 +448,10 @@ func (e *HostEnvironment) terminateRunningProcesses(ctx context.Context) {
return return
} }
script := buildWindowsWorkspaceKillScript(dirs) if killed, err := process.KillProcessesWithPaths(killCtx, dirs, e.TrackingID); err != nil {
logger.Debugf("workspace process cleanup reported errors: %v", err)
cmd := exec.CommandContext(killCtx, "powershell.exe", "-NoProfile", "-NonInteractive", "-Command", script)
out, err := cmd.CombinedOutput()
if err != nil {
logger.Debugf("workspace process-tree kill via PowerShell failed: %v output=%s", err, strings.TrimSpace(string(out)))
}
// Win32_Process exposes no working directory, so the scan above misses a
// process that merely runs in a workspace dir while pinning a handle on it.
if killed, err := process.KillProcessesWithCWDUnder(killCtx, dirs); err != nil {
logger.Debugf("workspace process kill by working directory reported errors: %v", err)
} else if killed > 0 { } else if killed > 0 {
logger.Debugf("terminated %d leftover process(es) by workspace working directory", killed) logger.Debugf("terminated %d leftover workspace process(es)", killed)
} }
} }
@@ -492,26 +459,6 @@ func (e *HostEnvironment) terminateRunningProcesses(ctx context.Context) {
// wedge the runner slot. A var so tests can shrink it. // wedge the runner slot. A var so tests can shrink it.
var hostCleanupTimeout = 30 * time.Second var hostCleanupTimeout = 30 * time.Second
// runWithTimeout returns context.DeadlineExceeded once timeout elapses, leaking
// the goroutine: a delete blocked in a syscall (AV filter driver, dead network
// mount) cannot be interrupted, and leaking scratch state beats losing the
// runner's capacity slot forever. The idle stale-dir sweep reclaims it later.
func runWithTimeout(fn func(), timeout time.Duration) error {
done := make(chan struct{})
go func() {
defer close(done)
fn()
}()
timer := time.NewTimer(timeout)
defer timer.Stop()
select {
case <-done:
return nil
case <-timer.C:
return context.DeadlineExceeded
}
}
func (e *HostEnvironment) Remove() common.Executor { func (e *HostEnvironment) Remove() common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
logger := common.Logger(ctx) logger := common.Logger(ctx)
@@ -523,25 +470,14 @@ func (e *HostEnvironment) Remove() common.Executor {
} }
e.terminateRunningProcesses(ctx) e.terminateRunningProcesses(ctx)
// Removes per-job misc state only, never the toolcache root. Bounded because
// CleanUp is a caller-supplied, typically unbounded os.RemoveAll.
if e.CleanUp != nil {
logger.Debugf("running host environment cleanup callback")
if err := runWithTimeout(e.CleanUp, hostCleanupTimeout); err != nil {
logger.Warnf("host environment cleanup did not finish within %s; continuing job completion, scratch state may be leaked and is reclaimed by the idle stale-dir sweep", hostCleanupTimeout)
} else {
logger.Debugf("host environment cleanup callback finished")
}
}
// Detach: a cancelled ctx would skip removePathWithRetry's retries, // Detach: a cancelled ctx would skip removePathWithRetry's retries,
// which absorb Windows file-handle release lag after the kill above. // which absorb Windows file-handle release lag after the kill above.
rmCtx, rmCancel := context.WithTimeout(context.Background(), hostCleanupTimeout) rmCtx, rmCancel := context.WithTimeout(context.Background(), hostCleanupTimeout)
defer rmCancel() defer rmCancel()
var errs []error var errs []error
if err := removePathWithRetry(rmCtx, e.Path); err != nil { if err := removePathWithRetry(rmCtx, cmp.Or(e.Root, e.Path)); err != nil {
logger.Warnf("failed to remove host misc state %s: %v", e.Path, err) logger.Warnf("failed to remove host misc state %s: %v", cmp.Or(e.Root, e.Path), err)
errs = append(errs, err) errs = append(errs, err)
} }
if e.CleanWorkdir { if e.CleanWorkdir {
+35 -82
View File
@@ -161,10 +161,12 @@ func TestHostEnvironmentRemoveWorkdir(t *testing.T) {
Path: filepath.Join(miscRoot, "hostexecutor"), Path: filepath.Join(miscRoot, "hostexecutor"),
Workdir: workdir, Workdir: workdir,
CleanWorkdir: owned, CleanWorkdir: owned,
CleanUp: func() { _ = os.RemoveAll(miscRoot) }, Root: miscRoot,
} }
require.NoError(t, os.MkdirAll(e.Path, 0o700)) require.NoError(t, os.MkdirAll(e.Path, 0o700))
require.NoError(t, os.MkdirAll(filepath.Join(miscRoot, "tmp"), 0o700))
require.NoError(t, e.Remove()(t.Context())) require.NoError(t, e.Remove()(t.Context()))
assert.NoDirExists(t, miscRoot)
_, err := os.Stat(workdir) _, err := os.Stat(workdir)
if owned { if owned {
assert.ErrorIs(t, err, os.ErrNotExist) assert.ErrorIs(t, err, os.ErrNotExist)
@@ -201,87 +203,38 @@ func TestRemoveAllWithContextDoesNotHangOnStuckDelete(t *testing.T) {
} }
func TestHostEnvironmentRemoveDoesNotHang(t *testing.T) { func TestHostEnvironmentRemoveDoesNotHang(t *testing.T) {
for _, phase := range []string{"CleanUp callback", "path removal"} { synctest.Test(t, func(t *testing.T) {
t.Run(phase, func(t *testing.T) { origTimeout, origRemoveAll := hostCleanupTimeout, removeAll
synctest.Test(t, func(t *testing.T) { hostCleanupTimeout = 100 * time.Millisecond
origTimeout := hostCleanupTimeout release, stubDone := make(chan struct{}), make(chan struct{})
hostCleanupTimeout = 100 * time.Millisecond removeAll = func(string) error {
t.Cleanup(func() { hostCleanupTimeout = origTimeout }) defer close(stubDone)
release, stubDone := make(chan struct{}), make(chan struct{}) <-release
block := func() { return nil
defer close(stubDone) }
<-release t.Cleanup(func() {
} close(release)
e := &HostEnvironment{Path: filepath.Join(t.TempDir(), "misc", "hostexecutor"), StdOut: os.Stdout} <-stubDone
if phase == "CleanUp callback" { hostCleanupTimeout, removeAll = origTimeout, origRemoveAll
e.CleanUp = block
} else {
origRemoveAll := removeAll
removeAll = func(string) error {
block()
return nil
}
t.Cleanup(func() { removeAll = origRemoveAll })
}
t.Cleanup(func() {
close(release)
<-stubDone
})
require.NoError(t, os.MkdirAll(e.Path, 0o700))
done := make(chan error, 1)
go func() { done <- e.Remove()(common.WithLogger(t.Context(), logging.Default())) }()
select {
case err := <-done:
require.NoError(t, err)
case <-time.After(10 * time.Second):
t.Fatalf("Remove() hung on a stuck %s", phase)
}
})
}) })
e := &HostEnvironment{Path: filepath.Join(t.TempDir(), "misc", "hostexecutor"), StdOut: os.Stdout}
require.NoError(t, os.MkdirAll(e.Path, 0o700))
done := make(chan error, 1)
go func() { done <- e.Remove()(common.WithLogger(t.Context(), logging.Default())) }()
select {
case err := <-done:
require.NoError(t, err)
case <-time.After(10 * time.Second):
t.Fatal("Remove() hung on a stuck path removal")
}
})
}
func TestJoinPath(t *testing.T) {
assert.Equal(t, "//server/share/act/workflow/envs.txt", JoinPath("//server/share/act", "workflow/envs.txt"))
assert.Equal(t, "/var/run/act/workflow/envs.txt", JoinPath("/var/run/act/", "workflow", "envs.txt"))
for _, root := range []string{"//server/share/act", "/var/run/act", "C:/act"} {
assert.Equal(t, root+"/actions/key/main.go.out", JoinPath(root+"/actions/key", "main.go.out"))
assert.Equal(t, root+"/workflow", JoinPath(root+"/workflow/outputcmd.txt", ".."))
} }
} }
func TestBuildWindowsWorkspaceKillScript(t *testing.T) {
t.Run("single dir", func(t *testing.T) {
s := buildWindowsWorkspaceKillScript([]string{`C:\workspace\job1`})
assert.Contains(t, s, `$paths = @('C:\workspace\job1')`)
assert.Contains(t, s, "$selfPid = $PID")
assert.Contains(t, s, "$_.ProcessId -eq $selfPid")
assert.Contains(t, s, `$prefix = $p + '\'`)
assert.Contains(t, s, "$_.ExecutablePath.StartsWith($prefix")
assert.Contains(t, s, "$_.CommandLine.IndexOf($prefix")
assert.Contains(t, s, "taskkill.exe /PID $_.ProcessId /T /F")
})
t.Run("multiple dirs comma-separated", func(t *testing.T) {
s := buildWindowsWorkspaceKillScript([]string{
`C:\work\path`,
`C:\work\workdir`,
`C:\Users\runner\AppData\Local\Temp\job-42`,
})
assert.Contains(t, s, `'C:\work\path'`)
assert.Contains(t, s, `'C:\work\workdir'`)
assert.Contains(t, s, `'C:\Users\runner\AppData\Local\Temp\job-42'`)
assert.Contains(t, s, `'C:\work\path','C:\work\workdir',`)
})
t.Run("path with single quote is escaped", func(t *testing.T) {
s := buildWindowsWorkspaceKillScript([]string{`C:\work\it's\path`})
assert.Contains(t, s, `'C:\work\it''s\path'`)
assert.NotContains(t, s, `'C:\work\it's\path'`)
})
t.Run("path with wildcard metacharacters is matched literally", func(t *testing.T) {
s := buildWindowsWorkspaceKillScript([]string{`C:\work\[job]?1`})
assert.Contains(t, s, `'C:\work\[job]?1'`)
assert.NotContains(t, s, "-like")
assert.Contains(t, s, "StartsWith")
assert.Contains(t, s, "IndexOf")
})
t.Run("empty dir list still produces a valid script", func(t *testing.T) {
s := buildWindowsWorkspaceKillScript(nil)
assert.Contains(t, s, "$paths = @()")
assert.Contains(t, s, "Get-CimInstance Win32_Process")
})
}
@@ -0,0 +1,138 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package container
import (
"bufio"
"context"
"fmt"
"io"
"net"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"unsafe"
"github.com/stretchr/testify/require"
"golang.org/x/sys/windows"
)
func TestHostEnvironmentTrackingCleanup(t *testing.T) {
for _, tracking := range []string{"default", "", "shared-helper"} {
t.Run("tracking="+tracking, func(t *testing.T) {
ctx, cancel := context.WithTimeout(t.Context(), 30*time.Second)
defer cancel()
listener, err := net.Listen("tcp4", "127.0.0.1:0")
require.NoError(t, err)
defer listener.Close()
stop := context.AfterFunc(ctx, func() { _ = listener.Close() })
defer stop()
workspace := t.TempDir()
environment := &HostEnvironment{Path: t.TempDir(), TmpDir: workspace, Workdir: workspace, TrackingID: "gitea_test", StdOut: io.Discard}
env := map[string]string{}
for _, entry := range os.Environ() {
if name, value, ok := strings.Cut(entry, "="); ok && !strings.EqualFold(name, "RUNNER_TRACKING_ID") {
env[name] = value
}
}
env["GITEA_TRACKING_HELPER"] = "step"
env["GITEA_TRACKING_WORKSPACE"] = workspace
env["GITEA_TRACKING_ADDRESS"] = listener.Addr().String()
if tracking == "default" {
env["RUNNER_TRACKING_ID"] = environment.TrackingID
env["runner_tracking_id"] = "shared-helper"
} else {
env["runner_tracking_id"] = tracking
}
result := make(chan error, 1)
go func() {
result <- environment.Exec([]string{os.Args[0], "-test.run=^TestHostTrackingHelper$"}, env, "", workspace)(ctx)
}()
step, err := listener.Accept()
require.NoError(t, err)
defer step.Close()
require.NoError(t, step.SetDeadline(time.Now().Add(20*time.Second)))
var stepPID int
_, err = fmt.Fscanln(step, &stepPID)
require.NoError(t, err)
stepHandle, err := windows.OpenProcess(windows.PROCESS_QUERY_LIMITED_INFORMATION, false, uint32(stepPID))
require.NoError(t, err)
defer func() { _ = windows.CloseHandle(stepHandle) }()
require.Eventually(t, func() bool {
var assigned int32
result, _, _ := windows.NewLazySystemDLL("kernel32.dll").NewProc("IsProcessInJob").Call(uintptr(stepHandle), 0, uintptr(unsafe.Pointer(&assigned)))
return result != 0 && assigned != 0
}, 5*time.Second, 10*time.Millisecond)
_, err = step.Write([]byte{1})
require.NoError(t, err)
var helpers []windows.Handle
for range 2 {
connection, err := listener.Accept()
require.NoError(t, err)
t.Cleanup(func() { _ = connection.Close() })
require.NoError(t, connection.SetReadDeadline(time.Now().Add(20*time.Second)))
var pid int
_, err = fmt.Fscanln(connection, &pid)
require.NoError(t, err)
handle, err := windows.OpenProcess(windows.SYNCHRONIZE|windows.PROCESS_TERMINATE, false, uint32(pid))
require.NoError(t, err)
t.Cleanup(func() {
_ = windows.TerminateProcess(handle, 1)
_, _ = windows.WaitForSingleObject(handle, 5000)
_ = windows.CloseHandle(handle)
})
helpers = append(helpers, handle)
}
select {
case err := <-result:
require.NoError(t, err)
case <-ctx.Done():
t.Fatal(ctx.Err())
}
require.NoError(t, environment.Remove()(ctx))
for _, handle := range helpers {
if tracking == "default" {
result, err := windows.WaitForSingleObject(handle, 5000)
require.NoError(t, err)
require.EqualValues(t, windows.WAIT_OBJECT_0, result)
} else {
result, err := windows.WaitForSingleObject(handle, 0)
require.NoError(t, err)
require.EqualValues(t, windows.WAIT_TIMEOUT, result)
}
}
})
}
}
func TestHostTrackingHelper(t *testing.T) {
mode := os.Getenv("GITEA_TRACKING_HELPER")
if mode == "" {
return
}
connection, err := net.Dial("tcp4", os.Getenv("GITEA_TRACKING_ADDRESS"))
require.NoError(t, err)
_, err = fmt.Fprintln(connection, os.Getpid())
require.NoError(t, err)
if mode == "step" {
_, err := bufio.NewReader(connection).ReadByte()
require.NoError(t, err)
}
if mode != "descendant" {
child := exec.Command(os.Args[0], "-test.run=^TestHostTrackingHelper$", filepath.Join(os.Getenv("GITEA_TRACKING_WORKSPACE"), "tracking-reference"))
child.Env = append(os.Environ(), "GITEA_TRACKING_HELPER="+map[string]string{"step": "helper", "helper": "descendant"}[mode])
require.NoError(t, child.Start())
if mode == "step" {
require.NoError(t, connection.Close())
os.Exit(0)
}
}
_, err = io.Copy(io.Discard, connection)
require.NoError(t, err)
require.NoError(t, connection.Close())
os.Exit(0)
}
+1 -1
View File
@@ -557,7 +557,7 @@ func (c *kubernetesContainer) Copy(destPath string, files ...*FileEntry) common.
func (c *kubernetesContainer) CopyDir(destPath, srcPath string, useGitIgnore, skipGitDir bool) common.Executor { func (c *kubernetesContainer) CopyDir(destPath, srcPath string, useGitIgnore, skipGitDir bool) common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
return c.extract(ctx, destPath, true, func(writer io.Writer) error { // directories are actions and workspaces, megabytes of compressible sources return c.extract(ctx, destPath, true, func(writer io.Writer) error { // directories are actions and workspaces, megabytes of compressible sources
return writeDirTar(ctx, writer, "/", srcPath, useGitIgnore, skipGitDir, 0, 0) return writeDirTar(ctx, writer, "/", srcPath, useGitIgnore, skipGitDir, 0, 0, hostOS == "windows")
}) })
} }
} }
+8 -3
View File
@@ -21,7 +21,12 @@ import (
// parseEnvFile reads a GITHUB_ENV style file, splitting lines like GitHub's runner on Windows or elsewhere. // parseEnvFile reads a GITHUB_ENV style file, splitting lines like GitHub's runner on Windows or elsewhere.
func parseEnvFile(e Container, srcPath string, env *map[string]string, windows bool) common.Executor { func parseEnvFile(e Container, srcPath string, env *map[string]string, windows bool) common.Executor {
localEnv := *env return ParseEnvFile(e, srcPath, func(name, value string) {
(*env)[name] = value
}, windows)
}
func ParseEnvFile(e Container, srcPath string, setEnv func(name, value string), windows bool) common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
envTar, err := e.GetContainerArchive(ctx, srcPath) envTar, err := e.GetContainerArchive(ctx, srcPath)
if err != nil { if err != nil {
@@ -64,7 +69,7 @@ func parseEnvFile(e Container, srcPath string, env *map[string]string, windows b
multiLineEnv := strings.Index(line, "<<") multiLineEnv := strings.Index(line, "<<")
switch { switch {
case singleLineEnv != -1 && (multiLineEnv == -1 || singleLineEnv < multiLineEnv): case singleLineEnv != -1 && (multiLineEnv == -1 || singleLineEnv < multiLineEnv):
localEnv[line[:singleLineEnv]] = line[singleLineEnv+1:] setEnv(line[:singleLineEnv], line[singleLineEnv+1:])
case multiLineEnv != -1: case multiLineEnv != -1:
var multiLineEnvContent []string var multiLineEnvContent []string
multiLineEnvDelimiter := line[multiLineEnv+2:] multiLineEnvDelimiter := line[multiLineEnv+2:]
@@ -86,7 +91,7 @@ func parseEnvFile(e Container, srcPath string, env *map[string]string, windows b
if !delimiterFound { if !delimiterFound {
return fmt.Errorf("invalid format delimiter '%v' not found before end of file", multiLineEnvDelimiter) return fmt.Errorf("invalid format delimiter '%v' not found before end of file", multiLineEnvDelimiter)
} }
localEnv[line[:multiLineEnv]] = trimEnvLineEnding(strings.Join(multiLineEnvContent, ""), windows) setEnv(line[:multiLineEnv], trimEnvLineEnding(strings.Join(multiLineEnvContent, ""), windows))
default: default:
return fmt.Errorf("invalid format '%v', expected a line with '=' or '<<'", line) return fmt.Errorf("invalid format '%v', expected a line with '=' or '<<'", line)
} }
+14 -2
View File
@@ -34,6 +34,7 @@ func TestParseEnvFile(t *testing.T) {
want map[string]string want map[string]string
}{ }{
{"single line", "FOO=bar\nBAZ=qux\n", map[string]string{"FOO": "bar", "BAZ": "qux"}}, {"single line", "FOO=bar\nBAZ=qux\n", map[string]string{"FOO": "bar", "BAZ": "qux"}},
{"repeated names", "FOO=first\nfoo=middle\nFOO=last\n", map[string]string{"FOO": "last", "foo": "middle"}},
{"single line CRLF and final bare CR", "FOO=bar\r\nBAZ=qux\r", map[string]string{"FOO": "bar\r", "BAZ": "qux\r"}}, {"single line CRLF and final bare CR", "FOO=bar\r\nBAZ=qux\r", map[string]string{"FOO": "bar\r", "BAZ": "qux\r"}},
{"single line without final newline", "FOO=bar", map[string]string{"FOO": "bar"}}, {"single line without final newline", "FOO=bar", map[string]string{"FOO": "bar"}},
{"blank lines", "\nFOO=bar\n\n \nBAZ=qux\n\n", map[string]string{"FOO": "bar", "BAZ": "qux"}}, {"blank lines", "\nFOO=bar\n\n \nBAZ=qux\n\n", map[string]string{"FOO": "bar", "BAZ": "qux"}},
@@ -57,6 +58,17 @@ func TestParseEnvFile(t *testing.T) {
assert.Equal(t, test.want, env) assert.Equal(t, test.want, env)
}) })
} }
t.Run("ordered entries preserve repeated names", func(t *testing.T) {
hostEnv, envPath := newTestHostEnv(t)
require.NoError(t, os.WriteFile(envPath, []byte("FOO=first\r\nfoo<<EOF\r\nmiddle\r\nEOF\r\nFOO=last\r\n"), 0o600))
var entries [][2]string
require.NoError(t, ParseEnvFile(hostEnv, envPath, func(name, value string) {
entries = append(entries, [2]string{name, value})
}, true)(t.Context()))
assert.Equal(t, [][2]string{
{"FOO", "first"}, {"foo", "middle"}, {"FOO", "last"},
}, entries)
})
t.Run("Windows line endings", func(t *testing.T) { t.Run("Windows line endings", func(t *testing.T) {
hostEnv, envPath := newTestHostEnv(t) hostEnv, envPath := newTestHostEnv(t)
require.NoError(t, os.WriteFile(envPath, []byte("FOO=bar\r\nMULTI<<EOF\r\nline1\r\nline2\nEOF\r\n"), 0o600)) require.NoError(t, os.WriteFile(envPath, []byte("FOO=bar\r\nMULTI<<EOF\r\nline1\r\nline2\nEOF\r\n"), 0o600))
@@ -86,13 +98,13 @@ func TestParseEnvFileInvalidHeredoc(t *testing.T) {
} { } {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
e, envPath := newTestHostEnv(t) e, envPath := newTestHostEnv(t)
require.NoError(t, os.WriteFile(envPath, []byte(content), 0o600)) require.NoError(t, os.WriteFile(envPath, []byte("APPLIED=before-error\n"+content), 0o600))
env := map[string]string{} env := map[string]string{}
err := parseEnvFile(e, envPath, &env, false)(t.Context()) err := parseEnvFile(e, envPath, &env, false)(t.Context())
require.Error(t, err) require.Error(t, err)
assert.Contains(t, err.Error(), "delimiter") assert.Contains(t, err.Error(), "delimiter")
assert.Empty(t, env) assert.Equal(t, map[string]string{"APPLIED": "before-error"}, env)
}) })
} }
} }
+2 -2
View File
@@ -36,7 +36,7 @@ func writeFilesTar(ctx context.Context, w io.Writer, uid, gid int, files ...*Fil
} }
// writeDirTar prefixes entries with dstPath for extraction at root. // writeDirTar prefixes entries with dstPath for extraction at root.
func writeDirTar(ctx context.Context, w io.Writer, dstPath, srcPath string, useGitIgnore, skipGitDir bool, uid, gid int) error { func writeDirTar(ctx context.Context, w io.Writer, dstPath, srcPath string, useGitIgnore, skipGitDir bool, uid, gid int, executable bool) error {
logger := common.Logger(ctx) logger := common.Logger(ctx)
tw := tar.NewWriter(w) tw := tar.NewWriter(w)
srcPrefix := strings.TrimSuffix(filepath.Dir(srcPath), string(filepath.Separator)) + string(filepath.Separator) srcPrefix := strings.TrimSuffix(filepath.Dir(srcPath), string(filepath.Separator)) + string(filepath.Separator)
@@ -47,7 +47,7 @@ func writeDirTar(ctx context.Context, w io.Writer, dstPath, srcPath string, useG
SrcPrefix: srcPrefix, SrcPrefix: srcPrefix,
SkipGitDir: skipGitDir, SkipGitDir: skipGitDir,
UseGitIgnore: useGitIgnore, UseGitIgnore: useGitIgnore,
Handler: &filecollector.TarCollector{TarWriter: tw, UID: uid, GID: gid, DstDir: dstPath[1:]}, Handler: &filecollector.TarCollector{TarWriter: tw, UID: uid, GID: gid, DstDir: dstPath[1:], Executable: executable},
}).CollectFiles(ctx)); err != nil { }).CollectFiles(ctx)); err != nil {
return err return err
} }
+9 -6
View File
@@ -27,22 +27,25 @@ type Handler interface {
} }
type TarCollector struct { type TarCollector struct {
TarWriter *tar.Writer TarWriter *tar.Writer
UID int UID int
GID int GID int
DstDir string DstDir string
Executable bool // Windows has no exec bit, so mark everything executable like docker build
} }
func (tc TarCollector) WriteFile(fpath string, fi fs.FileInfo, linkName string, f io.Reader) error { func (tc TarCollector) WriteFile(fpath string, fi fs.FileInfo, linkName string, f io.Reader) error {
// create a new dir/file header // create a new dir/file header
header, err := tar.FileInfoHeader(fi, linkName) header, err := tar.FileInfoHeader(fi, filepath.ToSlash(linkName))
if err != nil { if err != nil {
return err return err
} }
// update the name to correctly reflect the desired destination when untaring // update the name to correctly reflect the desired destination when untaring
header.Name = path.Join(tc.DstDir, fpath) header.Name = path.Join(tc.DstDir, fpath)
header.Mode = int64(fi.Mode()) if tc.Executable {
header.Mode = header.Mode&0o755 | 0o111
}
header.Uid = tc.UID header.Uid = tc.UID
header.Gid = tc.GID header.Gid = tc.GID
+4
View File
@@ -54,6 +54,7 @@ func TestCollectFilesKeepsWhatGitTracksOrDoesNotIgnore(t *testing.T) {
} }
expected := map[string]string{".gitignore": "", "ign/tracked": "", "nested/kept": "", "renamed": "", "sub/.gitignore": "", "sub/kept": ""} expected := map[string]string{".gitignore": "", "ign/tracked": "", "nested/kept": "", "renamed": "", "sub/.gitignore": "", "sub/kept": ""}
if runtime.GOOS != "windows" { if runtime.GOOS != "windows" {
require.NoError(t, os.Chmod(filepath.Join(repoDir, "renamed"), 0o777|os.ModeSticky))
require.NoError(t, os.Symlink(".env", filepath.Join(repoDir, "test.env"))) require.NoError(t, os.Symlink(".env", filepath.Join(repoDir, "test.env")))
expected["test.env"] = ".env" expected["test.env"] = ".env"
} }
@@ -69,6 +70,9 @@ func TestCollectFilesKeepsWhatGitTracksOrDoesNotIgnore(t *testing.T) {
for header, err := tr.Next(); err != io.EOF; header, err = tr.Next() { for header, err := tr.Next(); err != io.EOF; header, err = tr.Next() {
require.NoError(t, err) require.NoError(t, err)
files[header.Name] = header.Linkname files[header.Name] = header.Linkname
if header.Name == "renamed" && runtime.GOOS != "windows" {
assert.Equal(t, int64(0o1777), header.Mode)
}
} }
assert.Contains(t, files, ".git/HEAD") assert.Contains(t, files, ".git/HEAD")
maps.DeleteFunc(files, func(name, _ string) bool { return strings.Contains("/"+name, "/.git/") }) maps.DeleteFunc(files, func(name, _ string) bool { return strings.Contains("/"+name, "/.git/") })
+25
View File
@@ -0,0 +1,25 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package lookpath
import (
"path/filepath"
"strings"
)
func lookPathWindows(file string, env map[string]string, lookup func(string, map[string]string) (string, error)) (string, error) {
found, err := lookup(file, env)
isBash := strings.EqualFold(file, "bash") || strings.EqualFold(file, "bash.exe")
if !isBash || err == nil && !strings.EqualFold(filepath.Base(filepath.Dir(found)), "System32") { // System32's bash is the WSL launcher, Git's bash replaces it
return found, err
}
if git, gitErr := lookup("git.exe", env); gitErr == nil {
for _, relative := range []string{"../bin/bash.exe", "../usr/bin/bash.exe", "bash.exe"} {
if bash, bashErr := lookup(filepath.Join(filepath.Dir(git), relative), env); bashErr == nil {
return bash, nil
}
}
}
return found, err
}
+56
View File
@@ -0,0 +1,56 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package lookpath
import (
"io/fs"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestLookPathWindowsFallsBackToGitBash(t *testing.T) {
root := "C:" + string(filepath.Separator)
system32 := filepath.Join(root, "Windows", "System32")
gitCmd, gitBin := filepath.Join(root, "Git", "cmd"), filepath.Join(root, "Git", "bin")
msys := filepath.Join(root, "msys64", "usr", "bin")
for _, test := range []struct {
name, command, want string
path []string
}{
{"wsl launcher first", "bash", filepath.Join(gitBin, "bash.exe"), []string{system32, gitCmd}},
{"no bash on path", "BASH.EXE", filepath.Join(gitBin, "bash.exe"), []string{gitCmd}},
{"own bash first", "bash", filepath.Join(msys, "bash.exe"), []string{msys, system32, gitCmd}},
{"no git", "bash", filepath.Join(system32, "bash.exe"), []string{system32}},
{"other command", "sh", filepath.Join(system32, "sh.exe"), []string{system32, gitCmd}},
} {
t.Run(test.name, func(t *testing.T) {
files := map[string]bool{
filepath.Join(system32, "bash.exe"): true, filepath.Join(system32, "sh.exe"): true,
filepath.Join(gitCmd, "git.exe"): true, filepath.Join(gitBin, "bash.exe"): true, filepath.Join(msys, "bash.exe"): true,
}
lookup := func(file string, env map[string]string) (string, error) {
candidates := []string{file}
if !strings.ContainsAny(file, `:\/`) {
candidates = nil
for directory := range strings.SplitSeq(env["PATH"], ";") {
candidates = append(candidates, filepath.Join(directory, strings.TrimSuffix(strings.ToLower(file), ".exe")+".exe"))
}
}
for _, candidate := range candidates {
if files[filepath.Clean(candidate)] {
return filepath.Clean(candidate), nil
}
}
return "", fs.ErrNotExist
}
got, err := lookPathWindows(test.command, map[string]string{"PATH": strings.Join(test.path, ";")}, lookup)
require.NoError(t, err)
assert.Equal(t, test.want, got)
})
}
}
+4
View File
@@ -53,6 +53,10 @@ func findExecutable(file string, exts []string) (string, error) {
// a suitable candidate. // a suitable candidate.
// The result may be an absolute path or a path relative to the current directory. // The result may be an absolute path or a path relative to the current directory.
func LookPath2(file string, env map[string]string) (string, error) { func LookPath2(file string, env map[string]string) (string, error) {
return lookPathWindows(file, env, lookPath)
}
func lookPath(file string, env map[string]string) (string, error) {
var exts []string var exts []string
x := getenv(env, `PATHEXT`) x := getenv(env, `PATHEXT`)
if x != "" { if x != "" {
+11 -22
View File
@@ -198,7 +198,7 @@ func runActionImpl(step actionStep, actionDir string, remoteAction *remoteAction
if err := maybeCopyToActionDir(ctx, step, actionDir, actionPath, containerActionDir); err != nil { if err := maybeCopyToActionDir(ctx, step, actionDir, actionPath, containerActionDir); err != nil {
return err return err
} }
containerArgs := nodeActionCommand(path.Join(containerActionDir, action.Runs.Main)) containerArgs := nodeActionCommand(container.JoinPath(containerActionDir, action.Runs.Main))
logger.Debugf("executing remote job container: %s", containerArgs) logger.Debugf("executing remote job container: %s", containerArgs)
rc.ApplyExtraPath(ctx, step.getEnv()) rc.ApplyExtraPath(ctx, step.getEnv())
@@ -226,7 +226,7 @@ func runActionImpl(step actionStep, actionDir string, remoteAction *remoteAction
execFileName := action.Runs.Main + ".out" execFileName := action.Runs.Main + ".out"
return common.NewPipelineExecutor( return common.NewPipelineExecutor(
rc.JobContainer.Exec([]string{"go", "build", "-o", execFileName, action.Runs.Main}, *step.getEnv(), "", containerActionDir), rc.JobContainer.Exec([]string{"go", "build", "-o", execFileName, action.Runs.Main}, *step.getEnv(), "", containerActionDir),
rc.JobContainer.Exec([]string{filepath.Join(containerActionDir, execFileName)}, *step.getEnv(), "", ""), rc.JobContainer.Exec([]string{container.JoinPath(containerActionDir, execFileName)}, *step.getEnv(), "", ""),
)(ctx) )(ctx)
default: default:
return fmt.Errorf("the runs.using key must be one of: %v, got %s", []string{ return fmt.Errorf("the runs.using key must be one of: %v, got %s", []string{
@@ -295,6 +295,9 @@ func (rc *RunContext) dockerActionImage(ctx context.Context, step actionStep, ac
func execAsDocker(ctx context.Context, step actionStep, actionName, actionDir, basedir string, localAction bool, stage stepStage) error { func execAsDocker(ctx context.Context, step actionStep, actionName, actionDir, basedir string, localAction bool, stage stepStage) error {
logger := common.Logger(ctx) logger := common.Logger(ctx)
rc := step.getRunContext() rc := step.getRunContext()
if rc.isWindowsHost() {
return errWindowsHostContainer
}
action := step.getActionModel() action := step.getActionModel()
if rc.kubernetes { if rc.kubernetes {
return errKubernetesDocker return errKubernetesDocker
@@ -525,22 +528,8 @@ func populateEnvsFromInput(ctx context.Context, step step, action *model.Action)
return nil return nil
} }
// lookupFold finds name in values, ignoring case where the environment does, as GitHub's inputs always do.
func lookupFold[V any](values map[string]V, name string, caseInsensitive bool) (V, bool) {
if value, ok := values[name]; ok || !caseInsensitive {
return value, ok
}
for key, value := range values {
if strings.EqualFold(key, name) {
return value, true
}
}
var zero V
return zero, false
}
func stepInput(step *model.Step, name string) string { func stepInput(step *model.Step, name string) string {
value, _ := lookupFold(step.With, name, true) value, _ := common.LookupFold(step.With, name, true)
return value return value
} }
@@ -548,7 +537,7 @@ func warnUnexpectedInputs(ctx context.Context, action *model.Action, inputs map[
docker := action.Runs.Using.IsDocker() docker := action.Runs.Using.IsDocker()
var unexpectedInputs []string var unexpectedInputs []string
for input := range inputs { for input := range inputs {
_, declared := lookupFold(action.Inputs, input, true) _, declared := common.LookupFold(action.Inputs, input, true)
containerInput := docker && (strings.EqualFold(input, "entryPoint") || strings.EqualFold(input, "args")) containerInput := docker && (strings.EqualFold(input, "entryPoint") || strings.EqualFold(input, "args"))
if !declared && !containerInput { if !declared && !containerInput {
unexpectedInputs = append(unexpectedInputs, input) unexpectedInputs = append(unexpectedInputs, input)
@@ -665,7 +654,7 @@ func runPreStep(step actionStep) common.Executor {
return err return err
} }
containerArgs := nodeActionCommand(path.Join(containerActionDir, action.Runs.Pre)) containerArgs := nodeActionCommand(container.JoinPath(containerActionDir, action.Runs.Pre))
logger.Debugf("executing remote job container: %s", containerArgs) logger.Debugf("executing remote job container: %s", containerArgs)
rc.ApplyExtraPath(ctx, step.getEnv()) rc.ApplyExtraPath(ctx, step.getEnv())
@@ -697,7 +686,7 @@ func runPreStep(step actionStep) common.Executor {
execFileName := action.Runs.Pre + ".out" execFileName := action.Runs.Pre + ".out"
return common.NewPipelineExecutor( return common.NewPipelineExecutor(
rc.JobContainer.Exec([]string{"go", "build", "-o", execFileName, action.Runs.Pre}, *step.getEnv(), "", containerActionDir), rc.JobContainer.Exec([]string{"go", "build", "-o", execFileName, action.Runs.Pre}, *step.getEnv(), "", containerActionDir),
rc.JobContainer.Exec([]string{filepath.Join(containerActionDir, execFileName)}, *step.getEnv(), "", ""), rc.JobContainer.Exec([]string{container.JoinPath(containerActionDir, execFileName)}, *step.getEnv(), "", ""),
)(ctx) )(ctx)
default: default:
return nil return nil
@@ -765,7 +754,7 @@ func runPostStep(step actionStep) common.Executor {
populateEnvsFromSavedState(step.getEnv(), step, rc) populateEnvsFromSavedState(step.getEnv(), step, rc)
containerArgs := nodeActionCommand(path.Join(containerActionDir, action.Runs.Post)) containerArgs := nodeActionCommand(container.JoinPath(containerActionDir, action.Runs.Post))
logger.Debugf("executing remote job container: %s", containerArgs) logger.Debugf("executing remote job container: %s", containerArgs)
rc.ApplyExtraPath(ctx, step.getEnv()) rc.ApplyExtraPath(ctx, step.getEnv())
@@ -794,7 +783,7 @@ func runPostStep(step actionStep) common.Executor {
execFileName := action.Runs.Post + ".out" execFileName := action.Runs.Post + ".out"
return common.NewPipelineExecutor( return common.NewPipelineExecutor(
rc.JobContainer.Exec([]string{"go", "build", "-o", execFileName, action.Runs.Post}, *step.getEnv(), "", containerActionDir), rc.JobContainer.Exec([]string{"go", "build", "-o", execFileName, action.Runs.Post}, *step.getEnv(), "", containerActionDir),
rc.JobContainer.Exec([]string{filepath.Join(containerActionDir, execFileName)}, *step.getEnv(), "", ""), rc.JobContainer.Exec([]string{container.JoinPath(containerActionDir, execFileName)}, *step.getEnv(), "", ""),
)(ctx) )(ctx)
default: default:
+35 -2
View File
@@ -115,7 +115,7 @@ runs:
"cwd": { "cwd": {
Description: "(Actual working directory)", Description: "(Actual working directory)",
Required: false, Required: false,
Default: "actionDir/actionPath", Default: filepath.Join("actionDir", "actionPath"),
}, },
"command": { "command": {
Description: "(Actual program)", Description: "(Actual program)",
@@ -144,7 +144,7 @@ runs:
} }
writeFile := func(filename string, _data []byte, perm fs.FileMode) error { writeFile := func(filename string, _data []byte, perm fs.FileMode) error {
assert.Equal(t, "actionDir/actionPath/trampoline.js", filename) assert.Equal(t, filepath.Join("actionDir", "actionPath", "trampoline.js"), filename)
assert.Equal(t, fs.FileMode(0o644), perm) assert.Equal(t, fs.FileMode(0o644), perm)
return nil return nil
} }
@@ -248,6 +248,39 @@ func TestActionRunner(t *testing.T) {
} }
} }
func TestGoActionUsesLinuxExecutablePaths(t *testing.T) {
for _, stage := range []string{"main", "pre", "post"} {
t.Run(stage, func(t *testing.T) {
cm := &containerMock{}
containerActionDir := "/var/run/act/actions/" + model.UsesHash("/org/repo@ref")
cm.On("CopyDir", containerActionDir+"/", mock.Anything, false, true).Return(noopExecutor).Maybe()
cm.On("Exec", []string{"go", "build", "-o", stage + ".go.out", stage + ".go"}, mock.Anything, "", containerActionDir).Return(noopExecutor).Once()
cm.On("Exec", []string{containerActionDir + "/" + stage + ".go.out"}, mock.Anything, "", "").Return(noopExecutor).Once()
step := &stepActionRemote{
Step: &model.Step{ID: "step", Uses: "org/repo@ref"},
RunContext: &RunContext{
Config: &Config{NoActionPatch: true, PresetGitHubContext: &model.GithubContext{}},
Run: &model.Run{JobID: "job", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job": {}}}},
JobContainer: cm,
},
action: &model.Action{Runs: model.ActionRuns{Using: model.ActionRunsUsingGo, Main: "main.go", Pre: "pre.go", Post: "post.go"}},
actionDir: t.TempDir(),
env: map[string]string{},
remoteAction: mustNewRemoteAction(t, "org/repo@ref"),
}
switch stage {
case "main":
require.NoError(t, runActionImpl(step, step.actionDir, step.remoteAction)(t.Context()))
case "pre":
require.NoError(t, runPreStep(step)(t.Context()))
case "post":
require.NoError(t, runPostStep(step)(t.Context()))
}
cm.AssertExpectations(t)
})
}
}
func TestNodeActionCommandPreservesSymlinkedEntrypoint(t *testing.T) { func TestNodeActionCommandPreservesSymlinkedEntrypoint(t *testing.T) {
if runtime.GOOS == "windows" { if runtime.GOOS == "windows" {
t.Skip("symlink creation requires privileges on Windows") t.Skip("symlink creation requires privileges on Windows")
+1 -1
View File
@@ -165,7 +165,7 @@ func (rc *RunContext) refuseUnsecureCommand(ctx context.Context, command string)
} }
caseInsensitive := rc.JobContainer != nil && rc.JobContainer.IsEnvironmentCaseInsensitive() caseInsensitive := rc.JobContainer != nil && rc.JobContainer.IsEnvironmentCaseInsensitive()
for _, env := range []map[string]string{rc.currentStepEnv(), rc.Env, rc.GlobalEnv} { for _, env := range []map[string]string{rc.currentStepEnv(), rc.Env, rc.GlobalEnv} {
if value, ok := lookupFold(env, allowUnsecureCommandsVar, caseInsensitive); ok { if value, ok := common.LookupFold(env, allowUnsecureCommandsVar, caseInsensitive); ok {
if isTruthyEnv(value) { if isTruthyEnv(value) {
return false return false
} }
+1 -2
View File
@@ -10,7 +10,6 @@ import (
"errors" "errors"
"fmt" "fmt"
"maps" "maps"
"path"
"reflect" "reflect"
"regexp" "regexp"
"strings" "strings"
@@ -160,7 +159,7 @@ func getHashFilesFunction(ctx context.Context, rc *RunContext) func(v []reflect.
Name: name, Name: name,
Mode: 0o644, Mode: 0o644,
Body: hashfiles, Body: hashfiles,
}).Then(rc.JobContainer.Exec([]string{"node", path.Join(rc.JobContainer.GetActPath(), name)}, env, "", ""))(timeoutCtx) }).Then(rc.JobContainer.Exec([]string{"node", container.JoinPath(rc.JobContainer.GetActPath(), name)}, env, "", ""))(timeoutCtx)
quotedPatterns := expressionStringEscape(strings.Join(patterns, ", ")) quotedPatterns := expressionStringEscape(strings.Join(patterns, ", "))
switch { switch {
case err == nil: case err == nil:
+41 -18
View File
@@ -23,6 +23,7 @@ import (
"sync" "sync"
"time" "time"
"unicode" "unicode"
"unicode/utf8"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging" "gitea.com/gitea/runner/act/common/logging"
@@ -30,6 +31,8 @@ import (
"gitea.dev/actionslib/pkg/exprparser" "gitea.dev/actionslib/pkg/exprparser"
"gitea.dev/actionslib/pkg/model" "gitea.dev/actionslib/pkg/model"
unicodeencoding "golang.org/x/text/encoding/unicode"
"golang.org/x/text/transform"
) )
const ( const (
@@ -461,7 +464,7 @@ func evaluateJobEnv(ctx context.Context, rc *RunContext) (map[string]string, err
} }
} }
if workflowEnv != nil { if workflowEnv != nil {
evaluated = mergeMaps(workflowEnv, env) evaluated = rc.mergeEnv(workflowEnv, env)
evaluator = rc.NewExpressionEvaluatorWithEnv(ctx, evaluated) evaluator = rc.NewExpressionEvaluatorWithEnv(ctx, evaluated)
} }
var err error var err error
@@ -569,14 +572,34 @@ func tryUploadJobSummary(ctx context.Context, rc *RunContext) {
actPath := rc.JobContainer.GetActPath() actPath := rc.JobContainer.GetActPath()
// Reuse a single client across all step uploads so connections can be pooled. // Reuse a single client across all step uploads so connections can be pooled.
client := &http.Client{Timeout: jobSummaryUploadRequestTimeout, Transport: rc.Config.instanceTransportFor(base)} client := &http.Client{Timeout: jobSummaryUploadRequestTimeout, Transport: rc.Config.instanceTransportFor(base)}
maskers := rc.secretMaskers()
longestMask := 0
for i := 0; i < len(maskers); i += 2 {
longestMask = max(longestMask, len(maskers[i]))
}
masker := NewSecretReplacer(maskers)
limit := maxJobSummaryBytes + longestMask
for i := range rc.Run.Job().Steps { for i := range rc.Run.Job().Steps {
summaryPath := path.Join(actPath, "workflow", "step-summary-"+strconv.Itoa(i)+".md") summaryPath := container.JoinPath(actPath, "workflow", "step-summary-"+strconv.Itoa(i)+".md")
body, ok := readSingleFileFromContainerArchive(ctx, rc.JobContainer, summaryPath, maxJobSummaryBytes) body, ok := readSingleFileFromContainerArchive(ctx, rc.JobContainer, summaryPath, int64(limit+1))
if !ok || len(body) == 0 { if !ok || len(body) == 0 {
continue continue
} }
// Gitea renders summaries on the run page, so mask before the upload. masked := masker.Replace(string(body))
uploadJobSummary(ctx, client, base+strconv.Itoa(i)+"/summary", runtimeToken, []byte(rc.maskSecrets(string(body)))) split := len(body) > limit
if split { // the read can end inside a secret, so keep only what masks alike without the lookahead
unsplit := masker.Replace(string(body[:maxJobSummaryBytes]))
kept := 0
for kept < len(unsplit) && kept < len(masked) && unsplit[kept] == masked[kept] {
kept++
}
for kept > 0 && kept < len(masked) && !utf8.RuneStart(masked[kept]) {
kept--
}
masked = masked[:kept]
}
body = truncateJobSummary(ctx, []byte(masked), summaryPath, maxJobSummaryBytes, split)
uploadJobSummary(ctx, client, base+strconv.Itoa(i)+"/summary", runtimeToken, body)
} }
} }
@@ -690,26 +713,26 @@ func readSingleFileFromContainerArchive(ctx context.Context, env container.Execu
if entryName != requestedPath && entryName != path.Base(requestedPath) { if entryName != requestedPath && entryName != path.Base(requestedPath) {
continue continue
} }
// Summaries larger than the limit are truncated rather than dropped, so the b, err := io.ReadAll(io.LimitReader(transform.NewReader(tr, unicodeencoding.BOMOverride(unicodeencoding.UTF8.NewDecoder())), maxBytes))
// user still gets the leading content (mirroring how GitHub caps oversized
// step summaries instead of discarding them). Read one extra byte so an
// over-limit file is detected from the actual stream rather than trusting
// header.Size, then cap the returned content at maxBytes.
b, err := io.ReadAll(io.LimitReader(tr, maxBytes+1))
if err != nil { if err != nil {
return nil, false return nil, false
} }
if int64(len(b)) > maxBytes {
// Reserve room for the marker so the marked-up result still fits in maxBytes.
marker := []byte(jobSummaryTruncationMarker)
keep := max(maxBytes-int64(len(marker)), 0)
b = append(b[:keep], marker...)
common.Logger(ctx).Warnf("job summary truncated: path=%s max=%d", p, maxBytes)
}
return b, true return b, true
} }
} }
func truncateJobSummary(ctx context.Context, body []byte, summaryPath string, maxBytes int, split bool) []byte {
if !split && len(body) <= maxBytes {
return body
}
keep := max(min(len(body), maxBytes-len(jobSummaryTruncationMarker)), 0)
for keep > 0 && keep < len(body) && !utf8.RuneStart(body[keep]) {
keep--
}
common.Logger(ctx).Warnf("job summary truncated: path=%s max=%d", summaryPath, maxBytes)
return append(body[:keep], jobSummaryTruncationMarker...)
}
// useJobStageLogger tags the log entries of a job level executor with the stage they belong to. // useJobStageLogger tags the log entries of a job level executor with the stage they belong to.
func useJobStageLogger(stage string, executor common.Executor) common.Executor { func useJobStageLogger(stage string, executor common.Executor) common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
+87 -52
View File
@@ -34,6 +34,8 @@ import (
"github.com/stretchr/testify/mock" "github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
yaml "go.yaml.in/yaml/v4" yaml "go.yaml.in/yaml/v4"
"golang.org/x/text/encoding"
"golang.org/x/text/encoding/unicode"
) )
func TestJobExecutor(t *testing.T) { func TestJobExecutor(t *testing.T) {
@@ -932,49 +934,42 @@ func TestExtractJobIDFromRuntimeToken(t *testing.T) {
assert.Equal(t, int64(0), extractJobIDFromRuntimeToken("")) assert.Equal(t, int64(0), extractJobIDFromRuntimeToken(""))
} }
func TestReadSingleFileFromContainerArchiveFindsMatchingRegularFile(t *testing.T) { func TestReadSingleFileFromContainerArchive(t *testing.T) {
ctx := t.Context() retainedBytes := 200 - len(jobSummaryTruncationMarker)
cm := &containerMock{}
cm.On("GetContainerArchive", ctx, "/var/run/act/workflow/SUMMARY.md").Return(
io.NopCloser(bytes.NewReader(tarArchive(t,
tarEntry{name: "workflow", typeflag: tar.TypeDir},
tarEntry{name: "other.md", body: "wrong"},
tarEntry{name: "SUMMARY.md", body: "right"},
))),
nil,
).Once()
body, ok := readSingleFileFromContainerArchive(ctx, cm, "/var/run/act/workflow/SUMMARY.md", 1024)
assert.True(t, ok)
assert.Equal(t, []byte("right"), body)
cm.AssertExpectations(t)
}
func TestReadSingleFileFromContainerArchiveSizeLimit(t *testing.T) {
for name, tt := range map[string]struct { for name, tt := range map[string]struct {
content string content string
maxBytes int64 maxBytes int64
want string want string
}{ }{
"truncates oversized summary": {strings.Repeat("a", 300), 200, strings.Repeat("a", 200-len(jobSummaryTruncationMarker)) + jobSummaryTruncationMarker}, "truncates oversized summary without reading entire file": {strings.Repeat("a", 100000), 200, strings.Repeat("a", 200-len(jobSummaryTruncationMarker)) + jobSummaryTruncationMarker},
"keeps exact limit": {"abc", 3, "abc"}, "keeps exact limit": {"abc", 3, "abc"},
"strips UTF-8 BOM before limit": {"\xef\xbb\xbfabc", 3, "abc"},
"decodes UTF-16 before limit": {"\xff\xfea\x00b\x00c\x00", 3, "abc"},
"limits expanded UTF-16": {"\xff\xfe" + strings.Repeat("\x2d\x4e", 150), 200, strings.Repeat("中", retainedBytes/3) + jobSummaryTruncationMarker},
"keeps complete UTF-8 characters": {strings.Repeat("a", retainedBytes-1) + "€" + strings.Repeat("a", 200), 200, strings.Repeat("a", retainedBytes-1) + jobSummaryTruncationMarker},
} { } {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
logger, hook := logtest.NewLogger() logger, hook := logtest.NewLogger()
ctx := common.WithLogger(t.Context(), logger) ctx := common.WithLogger(t.Context(), logger)
cm := &containerMock{} cm := &containerMock{}
archive := bytes.NewReader(tarArchive(t,
tarEntry{name: "workflow", typeflag: tar.TypeDir},
tarEntry{name: "other.md", body: "wrong"},
tarEntry{name: "SUMMARY.md", body: tt.content},
))
cm.On("GetContainerArchive", ctx, "/var/run/act/workflow/SUMMARY.md").Return( cm.On("GetContainerArchive", ctx, "/var/run/act/workflow/SUMMARY.md").Return(
io.NopCloser(bytes.NewReader(tarArchive(t, tarEntry{name: "SUMMARY.md", body: tt.content}))), io.NopCloser(archive),
nil, nil,
).Once() ).Once()
body, ok := readSingleFileFromContainerArchive(ctx, cm, "/var/run/act/workflow/SUMMARY.md", tt.maxBytes) body, ok := readSingleFileFromContainerArchive(ctx, cm, "/var/run/act/workflow/SUMMARY.md", tt.maxBytes+1)
assert.LessOrEqual(t, int64(len(body)), tt.maxBytes+1)
body = truncateJobSummary(ctx, body, "/var/run/act/workflow/SUMMARY.md", int(tt.maxBytes), false)
assert.True(t, ok) assert.True(t, ok)
assert.Equal(t, []byte(tt.want), body) assert.Equal(t, []byte(tt.want), body)
assert.LessOrEqual(t, int64(len(body)), tt.maxBytes) assert.LessOrEqual(t, int64(len(body)), tt.maxBytes)
if int64(len(tt.content)) > tt.maxBytes { if strings.HasSuffix(tt.want, jobSummaryTruncationMarker) {
if assert.Len(t, hook.AllRecords(), 1) { if assert.Len(t, hook.AllRecords(), 1) {
assert.Contains(t, hook.LastRecord().Message, "job summary truncated") assert.Contains(t, hook.LastRecord().Message, "job summary truncated")
} }
@@ -1113,35 +1108,75 @@ func TestJobSetContinueOnError(t *testing.T) {
} }
func TestTryUploadJobSummaryMasksSecrets(t *testing.T) { func TestTryUploadJobSummaryMasksSecrets(t *testing.T) {
var got string const prefix = "deployed true with s3cr3t and runtime-added via pr0xypw 😀"
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { const maskedPrefix = "deployed true with *** and *** via *** 😀"
body, err := io.ReadAll(r.Body) padding := strings.Repeat("a", maxJobSummaryBytes-len(jobSummaryTruncationMarker)-len(prefix)-2)
assert.NoError(t, err) content := prefix + padding + "s3cr3t" + strings.Repeat("中", 100)
got = string(body) want := maskedPrefix + padding + "***" + strings.Repeat("中", (len(prefix)-len(maskedPrefix)-1)/3) + jobSummaryTruncationMarker
w.WriteHeader(http.StatusNoContent) longSecret := "PREFIX" + strings.Repeat("xy", 100) + "Z"
})) readLimit := maxJobSummaryBytes + base64.StdEncoding.EncodedLen(len(longSecret)) + 1
defer server.Close() shrinkingPrefix := strings.Repeat(longSecret, (readLimit-len(longSecret)+1)/len(longSecret))
for name, testcase := range map[string]struct {
encoding encoding.Encoding
content, want string
masks []string
}{
"UTF-8": {unicode.UTF8, content, want, nil},
"UTF-8 BOM": {unicode.UTF8BOM, content, want, nil},
"UTF-16LE BOM": {unicode.UTF16(unicode.LittleEndian, unicode.UseBOM), content, want, nil},
"UTF-16BE BOM": {unicode.UTF16(unicode.BigEndian, unicode.UseBOM), content, want, nil},
"secret split by the read": {unicode.UTF8, strings.Repeat("s3cr3t", maxJobSummaryBytes/3), "", nil},
"split read keeps every masked secret before the limit": {
unicode.UTF8,
"# public\n" + strings.Repeat(longSecret, readLimit/len(longSecret)+1),
"# public\n" + strings.Repeat("***", (maxJobSummaryBytes-len("# public\n"))/len(longSecret)) + jobSummaryTruncationMarker,
[]string{longSecret},
},
"nested secret expands the split tail": {
unicode.UTF8,
shrinkingPrefix + strings.Repeat("*", readLimit-len(shrinkingPrefix)-len(longSecret)+1) + longSecret,
"",
[]string{"xy", longSecret},
},
} {
t.Run(name, func(t *testing.T) {
var got string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(r.Body)
assert.NoError(t, err)
got = string(body)
w.WriteHeader(http.StatusNoContent)
}))
defer server.Close()
cm := &containerMock{} content, err := testcase.encoding.NewEncoder().Bytes([]byte(testcase.content))
cm.On("GetContainerArchive", mock.Anything, "/var/run/act/workflow/step-summary-0.md").Return( require.NoError(t, err)
io.NopCloser(bytes.NewReader(tarArchive(t, tarEntry{ cm := &containerMock{}
name: "step-summary-0.md", body: "deployed true with s3cr3t and runtime-added via pr0xypw", cm.On("GetContainerArchive", mock.Anything, "/var/run/act/workflow/step-summary-0.md").Return(
}))), io.NopCloser(bytes.NewReader(tarArchive(t, tarEntry{
nil, name: "step-summary-0.md", body: string(content),
).Once() }))),
nil,
).Once()
rc := newJobSummaryRC(map[string]string{ rc := newJobSummaryRC(map[string]string{
"GITEA_ACTIONS_CAPABILITIES": "job-summary", "GITEA_ACTIONS_CAPABILITIES": "job-summary",
"ACTIONS_RUNTIME_URL": server.URL, "ACTIONS_RUNTIME_URL": server.URL,
"ACTIONS_RUNTIME_TOKEN": fakeRuntimeToken(34), "ACTIONS_RUNTIME_TOKEN": fakeRuntimeToken(34),
"GITEA_RUN_ID": "12", "GITEA_RUN_ID": "12",
}, cm, 1) }, cm, 1)
rc.Config.Secrets = map[string]string{"TOK": "s3cr3t", "ACTIONS_STEP_DEBUG": "true"} rc.Config.Secrets = map[string]string{"TOK": "s3cr3t", "ACTIONS_STEP_DEBUG": "true"}
rc.Config.ExtraMasks = []string{"pr0xypw"} rc.Config.ExtraMasks = append([]string{"pr0xypw"}, testcase.masks...)
rc.Masks = []string{"runtime-added"} rc.Masks = []string{"runtime-added"}
tryUploadJobSummary(t.Context(), rc) tryUploadJobSummary(t.Context(), rc)
assert.Equal(t, "deployed true with *** and *** via ***", got) if testcase.want == "" {
cm.AssertExpectations(t) assert.Equal(t, jobSummaryTruncationMarker, strings.TrimLeft(got, "*"))
} else {
assert.Equal(t, testcase.want, got)
}
cm.AssertExpectations(t)
})
}
} }
+4 -4
View File
@@ -82,8 +82,8 @@ func (rc *RunContext) runJobHook(ctx context.Context, hookPath, name string) err
// the first one wrote. // the first one wrote.
func (rc *RunContext) setupHookFileCommands(ctx context.Context, env map[string]string) error { func (rc *RunContext) setupHookFileCommands(ctx context.Context, env map[string]string) error {
actPath := rc.JobContainer.GetActPath() actPath := rc.JobContainer.GetActPath()
env["GITHUB_ENV"] = path.Join(actPath, hookEnvFileCommand) env["GITHUB_ENV"] = container.JoinPath(actPath, hookEnvFileCommand)
env["GITHUB_PATH"] = path.Join(actPath, hookPathFileCommand) env["GITHUB_PATH"] = container.JoinPath(actPath, hookPathFileCommand)
aliasGiteaEnv(env) aliasGiteaEnv(env)
return rc.JobContainer.Copy(actPath, return rc.JobContainer.Copy(actPath,
@@ -94,7 +94,7 @@ func (rc *RunContext) setupHookFileCommands(ctx context.Context, env map[string]
func (rc *RunContext) processHookFileCommands(ctx context.Context) error { func (rc *RunContext) processHookFileCommands(ctx context.Context) error {
err := processRunnerEnvFileCommand(ctx, hookEnvFileCommand, rc, rc.setEnvFile) err := processRunnerEnvFileCommand(ctx, hookEnvFileCommand, rc, rc.setEnvFile)
if pathErr := rc.UpdateExtraPath(ctx, path.Join(rc.JobContainer.GetActPath(), hookPathFileCommand)); pathErr != nil && err == nil { if pathErr := rc.UpdateExtraPath(ctx, container.JoinPath(rc.JobContainer.GetActPath(), hookPathFileCommand)); pathErr != nil && err == nil {
err = pathErr err = pathErr
} }
return err return err
@@ -109,7 +109,7 @@ func hookCommand(hookPath string) (cmd []string, shell string) {
case ".sh": case ".sh":
return []string{"bash", "-e", hookPath}, "bash -e {0}" return []string{"bash", "-e", hookPath}, "bash -e {0}"
case ".ps1": case ".ps1":
return []string{"pwsh", "-command", ". '" + hookPath + "'"}, `pwsh -command ". '{0}'"` return []string{"pwsh", "-command", ". '" + expressionStringEscape(hookPath) + "'"}, `pwsh -command ". '{0}'"`
default: default:
return []string{hookPath}, "" return []string{hookPath}, ""
} }
+4 -3
View File
@@ -246,9 +246,10 @@ func TestHookCommand(t *testing.T) {
cmd []string cmd []string
shell string shell string
}{ }{
"/hooks/started.sh": {[]string{"bash", "-e", "/hooks/started.sh"}, "bash -e {0}"}, "/hooks/started.sh": {[]string{"bash", "-e", "/hooks/started.sh"}, "bash -e {0}"},
"/hooks/started.PS1": {[]string{"pwsh", "-command", ". '/hooks/started.PS1'"}, `pwsh -command ". '{0}'"`}, "/hooks/started.PS1": {[]string{"pwsh", "-command", ". '/hooks/started.PS1'"}, `pwsh -command ". '{0}'"`},
"/hooks/started": {[]string{"/hooks/started"}, ""}, "/hooks/started": {[]string{"/hooks/started"}, ""},
`C:\hooks\O'Brien\started.ps1`: {[]string{"pwsh", "-command", `. 'C:\hooks\O''Brien\started.ps1'`}, `pwsh -command ". '{0}'"`},
} { } {
cmd, shell := hookCommand(hookPath) cmd, shell := hookCommand(hookPath)
assert.Equal(t, want.cmd, cmd, hookPath) assert.Equal(t, want.cmd, cmd, hookPath)
+5 -1
View File
@@ -479,11 +479,15 @@ func (f textFormatter) isColored() bool {
// such as a container name or a job summary. Masks added at runtime count, so a summary written // such as a container name or a job summary. Masks added at runtime count, so a summary written
// after ::add-mask:: is covered too. // after ::add-mask:: is covered too.
func (rc *RunContext) maskSecrets(value string) string { func (rc *RunContext) maskSecrets(value string) string {
return NewSecretReplacer(rc.secretMaskers()).Replace(value)
}
func (rc *RunContext) secretMaskers() []string {
oldnew := rc.Config.maskers() oldnew := rc.Config.maskers()
for _, mask := range rc.Masks { for _, mask := range rc.Masks {
oldnew = AppendSecretMasker(oldnew, mask) oldnew = AppendSecretMasker(oldnew, mask)
} }
return NewSecretReplacer(oldnew).Replace(value) return oldnew
} }
// maskers is every value this job's configuration says to hide, whatever the sink. // maskers is every value this job's configuration says to hide, whatever the sink.
+7 -5
View File
@@ -256,7 +256,7 @@ func TestActionScriptPaths(t *testing.T) {
} }
func TestPatchActionsAtTheContainerCopy(t *testing.T) { func TestPatchActionsAtTheContainerCopy(t *testing.T) {
copiedBundles := func(t *testing.T, noPatch bool) ([]string, []*container.FileEntry) { copiedBundles := func(t *testing.T, noPatch bool) ([]string, []*container.FileEntry, int64) {
t.Helper() t.Helper()
cm := &containerMock{} cm := &containerMock{}
@@ -274,6 +274,8 @@ func TestPatchActionsAtTheContainerCopy(t *testing.T) {
require.NoError(t, os.MkdirAll(filepath.Dir(script), 0o755)) require.NoError(t, os.MkdirAll(filepath.Dir(script), 0o755))
require.NoError(t, os.WriteFile(filepath.Join(filepath.Dir(script), "index.js"), []byte("require('./cache.js')"), 0o600)) require.NoError(t, os.WriteFile(filepath.Join(filepath.Dir(script), "index.js"), []byte("require('./cache.js')"), 0o600))
require.NoError(t, os.WriteFile(script, []byte(gateTSC), 0o600)) require.NoError(t, os.WriteFile(script, []byte(gateTSC), 0o600))
info, err := os.Stat(script)
require.NoError(t, err)
var calls []string var calls []string
var copied []*container.FileEntry var copied []*container.FileEntry
@@ -291,22 +293,22 @@ func TestPatchActionsAtTheContainerCopy(t *testing.T) {
source, err := os.ReadFile(script) source, err := os.ReadFile(script)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, gateTSC, string(source), "the shared copy is never written") assert.Equal(t, gateTSC, string(source), "the shared copy is never written")
return calls, copied return calls, copied, int64(info.Mode().Perm())
} }
t.Run("patched over the container's copy", func(t *testing.T) { t.Run("patched over the container's copy", func(t *testing.T) {
calls, copied := copiedBundles(t, false) calls, copied, mode := copiedBundles(t, false)
assert.Equal(t, []string{"CopyDir", "Copy"}, calls) assert.Equal(t, []string{"CopyDir", "Copy"}, calls)
require.Len(t, copied, 1) require.Len(t, copied, 1)
assert.Equal(t, "dist/cache.js", copied[0].Name) assert.Equal(t, "dist/cache.js", copied[0].Name)
assert.Equal(t, int64(0o600), copied[0].Mode) assert.Equal(t, mode, copied[0].Mode)
assert.True(t, gateOpened(copied[0].Body)) assert.True(t, gateOpened(copied[0].Body))
}) })
// The escape hatch, for an action the edit breaks: the artifact actions refuse again, and the // The escape hatch, for an action the edit breaks: the artifact actions refuse again, and the
// cache client keeps to v1. // cache client keeps to v1.
t.Run("as shipped when the runner is told not to patch", func(t *testing.T) { t.Run("as shipped when the runner is told not to patch", func(t *testing.T) {
calls, copied := copiedBundles(t, true) calls, copied, _ := copiedBundles(t, true)
assert.Equal(t, []string{"CopyDir"}, calls) assert.Equal(t, []string{"CopyDir"}, calls)
assert.Empty(t, copied) assert.Empty(t, copied)
}) })
+3 -2
View File
@@ -160,8 +160,9 @@ func TestCloneRemoteReusableWorkflowSendsInstanceTokenAndClientCert(t *testing.T
Run: &model.Run{JobID: "j1", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"j1": {}}}}, Run: &model.Run{JobID: "j1", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"j1": {}}}},
} }
require.Error(t, cloneRemoteReusableWorkflow(rc, server.URL+"/owner/repo", "main", t.TempDir())(t.Context())) err = cloneRemoteReusableWorkflow(rc, server.URL+"/owner/repo", "main", t.TempDir())(t.Context())
require.True(t, gotCert) require.Error(t, err)
require.True(t, gotCert, "clone failed before receiving a client certificate: %v", err)
require.Equal(t, "token", gotToken) require.Equal(t, "token", gotToken)
} }
+64 -16
View File
@@ -81,6 +81,7 @@ type RunContext struct {
jobRunDefaults model.RunDefaults // defaults.run, resolved once at job setup as GitHub does jobRunDefaults model.RunDefaults // defaults.run, resolved once at job setup as GitHub does
platformImage string // container.image or the runs-on pick, resolved once by isEnabled platformImage string // container.image or the runs-on pick, resolved once by isEnabled
jobIndex, jobTotal int // strategy.job-index and job-total, total 0 when unknown jobIndex, jobTotal int // strategy.job-index and job-total, total 0 when unknown
trackingID string // RUNNER_TRACKING_ID of the job, generated on first use
runtimeToken string // ACTIONS_RUNTIME_TOKEN of the workflow run, shared with its called workflows and composite actions runtimeToken string // ACTIONS_RUNTIME_TOKEN of the workflow run, shared with its called workflows and composite actions
jobEnv map[string]string // workflow and job env as evaluated for the container and service specs jobEnv map[string]string // workflow and job env as evaluated for the container and service specs
@@ -191,7 +192,7 @@ func (rc *RunContext) GetEnv() map[string]string {
if rc.Run != nil && rc.Run.Workflow != nil && rc.Config != nil { if rc.Run != nil && rc.Run.Workflow != nil && rc.Config != nil {
job := rc.Run.Job() job := rc.Run.Job()
if job != nil { if job != nil {
rc.Env = mergeMaps(rc.Run.Workflow.Env, job.Environment(), rc.Config.Env) rc.Env = rc.mergeEnv(rc.Run.Workflow.Env, job.Environment(), rc.Config.Env)
} }
} }
if rc.runtimeToken != "" { if rc.runtimeToken != "" {
@@ -429,27 +430,58 @@ func (rc *RunContext) startHostEnvironment() common.Executor {
Workdir: rc.Config.Workdir, Workdir: rc.Config.Workdir,
CleanWorkdir: rc.Config.CleanWorkdir, CleanWorkdir: rc.Config.CleanWorkdir,
ActPath: actPath, ActPath: actPath,
CleanUp: func() { Root: miscpath,
os.RemoveAll(miscpath) TrackingID: rc.runnerTrackingID(),
}, StdOut: logWriter,
StdOut: logWriter, AllocatePTY: rc.Config.AllocatePTY,
AllocatePTY: rc.Config.AllocatePTY,
} }
rc.cleanUpJobContainer = rc.JobContainer.Remove() rc.cleanUpJobContainer = rc.JobContainer.Remove()
rc.setRunnerContextEnv(ctx) rc.setRunnerContextEnv(ctx)
for _, env := range os.Environ() { rc.importHostEnv(os.Environ())
if k, v, ok := strings.Cut(env, "="); ok {
// don't override
if _, ok := rc.Env[k]; !ok {
rc.Env[k] = v
}
}
}
return rc.copyWorkflowFiles()(ctx) return rc.copyWorkflowFiles()(ctx)
} }
} }
func (rc *RunContext) runnerTrackingID() string {
job := rc.topLevelRunContext()
if job.trackingID == "" {
job.trackingID = "gitea_" + rand.Text()
}
return job.trackingID
}
func (rc *RunContext) isEnvCaseInsensitive() bool {
if rc.JobContainer != nil {
return rc.JobContainer.IsEnvironmentCaseInsensitive()
}
return runtime.GOOS == "windows" && rc.IsHostEnv()
}
func (rc *RunContext) importHostEnv(environ []string) {
for _, env := range environ {
if key, value, ok := strings.Cut(env, "="); ok {
if key == "RUNNER_TRACKING_ID" || rc.isEnvCaseInsensitive() && strings.EqualFold(key, "RUNNER_TRACKING_ID") {
continue
}
_, exists := common.LookupFold(rc.Env, key, rc.isEnvCaseInsensitive())
if !exists {
rc.Env[key] = value
}
}
}
}
func (rc *RunContext) mergeEnv(envs ...map[string]string) map[string]string {
if rc.isEnvCaseInsensitive() {
env := map[string]string{}
mergeIntoMapCaseInsensitive(env, envs...)
return env
} else {
return mergeMaps(envs...)
}
}
func (rc *RunContext) setRunnerContextEnv(ctx context.Context) { func (rc *RunContext) setRunnerContextEnv(ctx context.Context) {
for k, v := range rc.getRunnerContext(ctx) { for k, v := range rc.getRunnerContext(ctx) {
if v, ok := v.(string); ok { if v, ok := v.(string); ok {
@@ -460,6 +492,15 @@ func (rc *RunContext) setRunnerContextEnv(ctx context.Context) {
var errKubernetesDocker = errors.New("docker actions and docker:// steps are not supported on Kubernetes") var errKubernetesDocker = errors.New("docker actions and docker:// steps are not supported on Kubernetes")
var errWindowsHostContainer = errors.New("container actions and docker:// steps are not supported in Windows host jobs")
var hostOS = runtime.GOOS // replaced in tests
func (rc *RunContext) isWindowsHost() bool {
_, host := rc.JobContainer.(*container.HostEnvironment)
return host && hostOS == "windows"
}
func (rc *RunContext) startKubernetesPod() common.Executor { func (rc *RunContext) startKubernetesPod() common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
if rc.containerSpec.Options != "" || len(rc.containerSpec.Volumes) > 0 || rc.containerSpec.Credentials != nil { if rc.containerSpec.Options != "" || len(rc.containerSpec.Volumes) > 0 || rc.containerSpec.Credentials != nil {
@@ -1311,6 +1352,10 @@ func (rc *RunContext) resolvePlatformImage(ctx context.Context) error {
} }
image, rc.kubernetes = strings.CutPrefix(image, "kubernetes://") image, rc.kubernetes = strings.CutPrefix(image, "kubernetes://")
rc.platformImage = cmp.Or(rc.containerSpec.Image, image) rc.platformImage = cmp.Or(rc.containerSpec.Image, image)
if rc.isEnvCaseInsensitive() {
rc.Env = nil // Rebuild scopes after the host platform is known.
rc.GetEnv()
}
return nil return nil
} }
@@ -1509,7 +1554,6 @@ func (rc *RunContext) getGithubContext(ctx context.Context) *model.GithubContext
RepositoryOwner: rc.Config.Env["GITHUB_REPOSITORY_OWNER"], RepositoryOwner: rc.Config.Env["GITHUB_REPOSITORY_OWNER"],
RetentionDays: cmp.Or(rc.Config.Env["GITHUB_RETENTION_DAYS"], "0"), RetentionDays: cmp.Or(rc.Config.Env["GITHUB_RETENTION_DAYS"], "0"),
RunnerPerflog: cmp.Or(rc.Config.Env["RUNNER_PERFLOG"], "/dev/null"), RunnerPerflog: cmp.Or(rc.Config.Env["RUNNER_PERFLOG"], "/dev/null"),
RunnerTrackingID: rc.Config.Env["RUNNER_TRACKING_ID"],
Repository: rc.Config.Env["GITHUB_REPOSITORY"], Repository: rc.Config.Env["GITHUB_REPOSITORY"],
Ref: rc.Config.Env["GITHUB_REF"], Ref: rc.Config.Env["GITHUB_REF"],
Sha: rc.Config.Env["SHA_REF"], Sha: rc.Config.Env["SHA_REF"],
@@ -1523,6 +1567,8 @@ func (rc *RunContext) getGithubContext(ctx context.Context) *model.GithubContext
ghc.EventPath = rc.JobContainer.GetActPath() + "/workflow/event.json" ghc.EventPath = rc.JobContainer.GetActPath() + "/workflow/event.json"
} }
ghc.RunnerTrackingID = rc.runnerTrackingID()
if preset := rc.Config.PresetGitHubContext; preset != nil { if preset := rc.Config.PresetGitHubContext; preset != nil {
merged := *preset merged := *preset
merged.Workflow, merged.Job, merged.Workspace, merged.EventPath = ghc.Workflow, ghc.Job, ghc.Workspace, ghc.EventPath merged.Workflow, merged.Job, merged.Workspace, merged.EventPath = ghc.Workflow, ghc.Job, ghc.Workspace, ghc.EventPath
@@ -1645,7 +1691,9 @@ func (rc *RunContext) withGithubEnv(ctx context.Context, github *model.GithubCon
env["GITHUB_REPOSITORY_OWNER"] = github.RepositoryOwner env["GITHUB_REPOSITORY_OWNER"] = github.RepositoryOwner
env["GITHUB_RETENTION_DAYS"] = github.RetentionDays env["GITHUB_RETENTION_DAYS"] = github.RetentionDays
env["RUNNER_PERFLOG"] = github.RunnerPerflog env["RUNNER_PERFLOG"] = github.RunnerPerflog
env["RUNNER_TRACKING_ID"] = github.RunnerTrackingID if _, ok := common.LookupFold(env, "RUNNER_TRACKING_ID", rc.isEnvCaseInsensitive()); !ok {
env["RUNNER_TRACKING_ID"] = github.RunnerTrackingID
}
env["GITHUB_BASE_REF"] = github.BaseRef env["GITHUB_BASE_REF"] = github.BaseRef
env["GITHUB_HEAD_REF"] = github.HeadRef env["GITHUB_HEAD_REF"] = github.HeadRef
env["GITHUB_SERVER_URL"] = github.ServerURL env["GITHUB_SERVER_URL"] = github.ServerURL
+50 -3
View File
@@ -19,6 +19,7 @@ import (
"path/filepath" "path/filepath"
"runtime" "runtime"
"slices" "slices"
"strconv"
"strings" "strings"
"sync" "sync"
"testing" "testing"
@@ -1082,6 +1083,51 @@ func TestRunContextGetEnv(t *testing.T) {
} }
} }
func TestRunContextEnvCaseSensitivity(t *testing.T) {
for _, caseInsensitive := range []bool{false, true} {
t.Run(strconv.FormatBool(caseInsensitive), func(t *testing.T) {
rc := createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, "runs-on: self-hosted\nenv:\n build_mode: job\n config_value: job", ""),
})
rc.Run.Workflow.Env = map[string]string{"BUILD_MODE": "workflow", "CONFIG_VALUE": "workflow", "PATH": "workflow-path", "runner_tool_cache": "workflow-cache"}
rc.Config.Env = map[string]string{"CONFIG_VALUE": "config", "RUNNER_TOOL_CACHE": "configured-cache"}
rc.Config.DisableActEnv, rc.Config.NoSkipCheckout = true, true
rc.Env = nil
rc.GetEnv()
rc.JobContainer = fakeContainer{ExecutionsEnvironment: &container.HostEnvironment{}, caseInsensitive: caseInsensitive}
require.NoError(t, rc.resolvePlatformImage(t.Context()))
if caseInsensitive {
assert.Equal(t, map[string]string{"build_mode": "job", "CONFIG_VALUE": "config", "PATH": "workflow-path", "RUNNER_TOOL_CACHE": "configured-cache"}, rc.GetEnv())
} else {
assert.Equal(t, map[string]string{
"BUILD_MODE": "workflow", "build_mode": "job", "CONFIG_VALUE": "config",
"config_value": "job", "PATH": "workflow-path", "runner_tool_cache": "workflow-cache", "RUNNER_TOOL_CACHE": "configured-cache",
}, rc.GetEnv())
}
want := maps.Clone(rc.GetEnv())
rc.Run.Workflow.Env = nil
require.NoError(t, rc.Run.Workflow.RawEnv.Encode(`${{ fromJSON('{"BUILD_MODE":"workflow","CONFIG_VALUE":"workflow","PATH":"workflow-path","runner_tool_cache":"workflow-cache"}') }}`))
rc.Env = nil
evaluated, err := evaluateJobEnv(t.Context(), rc)
require.NoError(t, err)
assert.Equal(t, want, evaluated)
rc.Env = evaluated
rc.importHostEnv([]string{"Path=host-path", "HOST_ONLY=value", "RUNNER_TRACKING_ID=parent", "runner_tracking_id=parent-lower"})
assert.NotContains(t, rc.Env, "RUNNER_TRACKING_ID")
assert.Equal(t, "workflow-path", rc.Env["PATH"])
assert.Equal(t, "value", rc.Env["HOST_ONLY"])
if caseInsensitive {
assert.NotContains(t, rc.Env, "Path")
assert.NotContains(t, rc.Env, "runner_tracking_id")
} else {
assert.Equal(t, "host-path", rc.Env["Path"])
assert.Equal(t, "parent-lower", rc.Env["runner_tracking_id"])
}
})
}
}
func TestSetActionEnvRefreshesInputs(t *testing.T) { func TestSetActionEnvRefreshesInputs(t *testing.T) {
rc := &RunContext{} rc := &RunContext{}
rc.setActionEnv(map[string]string{"INPUT_MSG": "pre"}) rc.setActionEnv(map[string]string{"INPUT_MSG": "pre"})
@@ -1405,13 +1451,14 @@ func TestGetJobContextReportsContainers(t *testing.T) {
} }
func TestCaptureJobContainerInfoExportsDockerWorkspace(t *testing.T) { func TestCaptureJobContainerInfoExportsDockerWorkspace(t *testing.T) {
workdir := t.TempDir()
job := &containerMock{} job := &containerMock{}
job.On("Inspect", mock.Anything).Return(&container.Info{ job.On("Inspect", mock.Anything).Return(&container.Info{
ID: "job-container-id", ID: "job-container-id",
Mounts: map[string]string{"/workspace": "/var/lib/docker/volumes/job/_data"}, Mounts: map[string]string{job.ToContainerPath(workdir): "/var/lib/docker/volumes/job/_data"},
}, nil) }, nil)
rc := &RunContext{ rc := &RunContext{
Config: &Config{Workdir: "/workspace/owner/repo/"}, Config: &Config{Workdir: filepath.Join(workdir, "owner", "repo") + string(filepath.Separator)},
Env: map[string]string{}, Env: map[string]string{},
JobContainer: job, JobContainer: job,
} }
@@ -1421,7 +1468,7 @@ func TestCaptureJobContainerInfoExportsDockerWorkspace(t *testing.T) {
assert.Equal(t, "job-container-id", rc.jobContainerID) assert.Equal(t, "job-container-id", rc.jobContainerID)
assert.Equal(t, "/var/lib/docker/volumes/job/_data/owner/repo", rc.Env["GITEA_DOCKER_WORKSPACE"]) assert.Equal(t, "/var/lib/docker/volumes/job/_data/owner/repo", rc.Env["GITEA_DOCKER_WORKSPACE"])
rc.Config.Workdir = "/elsewhere" rc.Config.Workdir = filepath.Dir(workdir)
rc.Env = map[string]string{} rc.Env = map[string]string{}
require.NoError(t, rc.captureJobContainerInfo()(t.Context())) require.NoError(t, rc.captureJobContainerInfo()(t.Context()))
assert.NotContains(t, rc.Env, "GITEA_DOCKER_WORKSPACE") assert.NotContains(t, rc.Env, "GITEA_DOCKER_WORKSPACE")
+43
View File
@@ -0,0 +1,43 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package runner
import (
"strings"
"testing"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestRunEventHostEnvironmentUTF8(t *testing.T) {
handler := logtest.New()
t.Cleanup(func() {
if t.Failed() {
for _, record := range handler.AllRecords() {
stepIDs, _ := record.Attr("stepID")
t.Logf("stepID=%v: %s", stepIDs, record.Message)
}
}
})
table := TestJobFileInfo{
workdir: workdir,
workflowPath: "windows-utf8",
eventName: "push",
platforms: map[string]string{"windows-latest": "-self-hosted"},
}
table.runTest(WithJobLoggerFactory(t.Context(), factoryFunc(func() logging.Handler { return handler })), t, &Config{Env: testCheckoutEnv()})
var outputSteps []string
for _, record := range handler.AllRecords() {
if strings.TrimRight(record.Message, "\r\n") == "你好,世界" {
stepIDs, ok := record.Attr("stepID")
require.True(t, ok)
outputSteps = append(outputSteps, stepIDs.([]string)...)
}
}
assert.Equal(t, []string{"cmd", "pwsh", "powershell"}, outputSteps)
}
+21 -19
View File
@@ -69,10 +69,15 @@ func (s stepStage) String() string {
} }
func processRunnerEnvFileCommand(ctx context.Context, fileName string, rc *RunContext, setter func(context.Context, map[string]string, string)) error { func processRunnerEnvFileCommand(ctx context.Context, fileName string, rc *RunContext, setter func(context.Context, map[string]string, string)) error {
if rc.JobContainer.IsEnvironmentCaseInsensitive() {
return container.ParseEnvFile(rc.JobContainer, container.JoinPath(rc.JobContainer.GetActPath(), fileName), func(name, value string) {
setter(ctx, map[string]string{"name": name}, value)
}, true)(ctx)
}
env := map[string]string{} env := map[string]string{}
err := rc.JobContainer.UpdateFromEnv(path.Join(rc.JobContainer.GetActPath(), fileName), &env)(ctx) err := rc.JobContainer.UpdateFromEnv(container.JoinPath(rc.JobContainer.GetActPath(), fileName), &env)(ctx)
for k, v := range env { for key, value := range env {
setter(ctx, map[string]string{"name": k}, v) setter(ctx, map[string]string{"name": key}, value)
} }
return err return err
} }
@@ -139,16 +144,16 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
actPath := rc.JobContainer.GetActPath() actPath := rc.JobContainer.GetActPath()
outputFileCommand := path.Join("workflow", "outputcmd.txt") outputFileCommand := path.Join("workflow", "outputcmd.txt")
(*step.getEnv())["GITHUB_OUTPUT"] = path.Join(actPath, outputFileCommand) (*step.getEnv())["GITHUB_OUTPUT"] = container.JoinPath(actPath, outputFileCommand)
stateFileCommand := path.Join("workflow", "statecmd.txt") stateFileCommand := path.Join("workflow", "statecmd.txt")
(*step.getEnv())["GITHUB_STATE"] = path.Join(actPath, stateFileCommand) (*step.getEnv())["GITHUB_STATE"] = container.JoinPath(actPath, stateFileCommand)
pathFileCommand := path.Join("workflow", "pathcmd.txt") pathFileCommand := path.Join("workflow", "pathcmd.txt")
(*step.getEnv())["GITHUB_PATH"] = path.Join(actPath, pathFileCommand) (*step.getEnv())["GITHUB_PATH"] = container.JoinPath(actPath, pathFileCommand)
envFileCommand := path.Join("workflow", "envs.txt") envFileCommand := path.Join("workflow", "envs.txt")
(*step.getEnv())["GITHUB_ENV"] = path.Join(actPath, envFileCommand) (*step.getEnv())["GITHUB_ENV"] = container.JoinPath(actPath, envFileCommand)
// Per-step summary file. Composite sub-steps share the outer job step's index // Per-step summary file. Composite sub-steps share the outer job step's index
// via the Parent chain so all writes from within a composite action accumulate // via the Parent chain so all writes from within a composite action accumulate
@@ -156,7 +161,7 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
topRC := rc.topLevelRunContext() topRC := rc.topLevelRunContext()
stepSummaryIndex := topRC.CurrentStepIndex stepSummaryIndex := topRC.CurrentStepIndex
summaryFileCommand := path.Join("workflow", "step-summary-"+strconv.Itoa(stepSummaryIndex)+".md") summaryFileCommand := path.Join("workflow", "step-summary-"+strconv.Itoa(stepSummaryIndex)+".md")
(*step.getEnv())["GITHUB_STEP_SUMMARY"] = path.Join(actPath, summaryFileCommand) (*step.getEnv())["GITHUB_STEP_SUMMARY"] = container.JoinPath(actPath, summaryFileCommand)
aliasGiteaEnv(*step.getEnv()) aliasGiteaEnv(*step.getEnv())
// Reset the per-phase file-command files. GITHUB_STEP_SUMMARY is intentionally // Reset the per-phase file-command files. GITHUB_STEP_SUMMARY is intentionally
@@ -203,7 +208,7 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
if fileErr := processRunnerEnvFileCommand(fileCommandCtx, outputFileCommand, rc, rc.setOutput); fileErr != nil && err == nil { if fileErr := processRunnerEnvFileCommand(fileCommandCtx, outputFileCommand, rc, rc.setOutput); fileErr != nil && err == nil {
err = fileErr err = fileErr
} }
if fileErr := rc.UpdateExtraPath(fileCommandCtx, path.Join(actPath, pathFileCommand)); fileErr != nil && err == nil { if fileErr := rc.UpdateExtraPath(fileCommandCtx, container.JoinPath(actPath, pathFileCommand)); fileErr != nil && err == nil {
err = fileErr err = fileErr
} }
} }
@@ -351,17 +356,14 @@ func mergeIntoMapCaseInsensitive(target map[string]string, maps ...map[string]st
for k := range target { for k := range target {
foldKeys[strings.ToLower(k)] = k foldKeys[strings.ToLower(k)] = k
} }
toKey := func(s string) string {
foldKey := strings.ToLower(s)
if k, ok := foldKeys[foldKey]; ok {
return k
}
foldKeys[foldKey] = s
return s
}
for _, m := range maps { for _, m := range maps {
for k, v := range m { for key, value := range m {
target[toKey(k)] = v foldKey := strings.ToLower(key)
if previous, ok := foldKeys[foldKey]; ok {
delete(target, previous)
}
foldKeys[foldKey] = key
target[key] = value
} }
} }
} }
+3
View File
@@ -52,6 +52,9 @@ func (sd *stepDocker) runUsesContainer() common.Executor {
step := sd.Step step := sd.Step
return func(ctx context.Context) error { return func(ctx context.Context) error {
if rc.isWindowsHost() {
return errWindowsHostContainer
}
if rc.kubernetes { if rc.kubernetes {
return errKubernetesDocker return errKubernetesDocker
} }
+30
View File
@@ -159,3 +159,33 @@ func TestStepDockerNewStepContainer(t *testing.T) {
}) })
} }
} }
func TestWindowsHostContainerActions(t *testing.T) {
for _, test := range []struct {
name, hostOS string
jobContainer container.ExecutionsEnvironment
wantError bool
}{
{name: "Windows host", hostOS: "windows", jobContainer: &container.HostEnvironment{}, wantError: true},
{name: "Linux host", hostOS: "linux", jobContainer: &container.HostEnvironment{}},
{name: "macOS host", hostOS: "darwin", jobContainer: &container.HostEnvironment{}},
{name: "Windows with Linux job container", hostOS: "windows", jobContainer: &containerMock{}},
} {
t.Run(test.name, func(t *testing.T) {
original := hostOS
hostOS = test.hostOS
t.Cleanup(func() { hostOS = original })
rc := &RunContext{JobContainer: test.jobContainer, kubernetes: true}
wantError := errKubernetesDocker.Error()
if test.wantError {
wantError = errWindowsHostContainer.Error()
}
t.Run("docker step", func(t *testing.T) {
require.EqualError(t, (&stepDocker{RunContext: rc}).runUsesContainer()(t.Context()), wantError)
})
t.Run("container action", func(t *testing.T) {
require.EqualError(t, execAsDocker(t.Context(), &stepActionRemote{RunContext: rc}, "action", "", "", false, stepStageMain), wantError)
})
})
}
}
+31 -2
View File
@@ -100,7 +100,7 @@ func printRunActionHeader(ctx context.Context, step *model.Step, action *model.A
maps.Copy(inputs, step.With) maps.Copy(inputs, step.With)
if action != nil { if action != nil {
for name := range action.Inputs { for name := range action.Inputs {
if _, provided := lookupFold(step.With, name, true); !provided { if _, provided := common.LookupFold(step.With, name, true); !provided {
inputs[name] = env[inputEnvKey(name)] inputs[name] = env[inputEnvKey(name)]
} }
} }
@@ -292,7 +292,7 @@ func (sr *stepRun) setupShellCommand(ctx context.Context) (name, script string,
name += ".sh" name += ".sh"
case "pwsh", "powershell": case "pwsh", "powershell":
name += ".ps1" name += ".ps1"
runPrepend = "$ErrorActionPreference = 'stop'" runPrepend = "$ErrorActionPreference = 'stop'\n$OutputEncoding = [System.Text.UTF8Encoding]::new($false)\ntry { [Console]::OutputEncoding = $OutputEncoding } catch {}"
runAppend = "if ((Test-Path -LiteralPath variable:/LASTEXITCODE)) { exit $LASTEXITCODE }" runAppend = "if ((Test-Path -LiteralPath variable:/LASTEXITCODE)) { exit $LASTEXITCODE }"
case "cmd": case "cmd":
name += ".cmd" name += ".cmd"
@@ -302,6 +302,12 @@ func (sr *stepRun) setupShellCommand(ctx context.Context) (name, script string,
} }
script = fmt.Sprintf("%s\n%s\n%s", runPrepend, script, runAppend) script = fmt.Sprintf("%s\n%s\n%s", runPrepend, script, runAppend)
switch shellCommand {
case "powershell":
script = "\ufeff" + script
case "cmd":
script = strings.ReplaceAll(strings.ReplaceAll(script, "\r\n", "\n"), "\n", "\r\n")
}
if !strings.Contains(script, "::add-mask::") && !sr.RunContext.Config.InsecureSecrets { if !strings.Contains(script, "::add-mask::") && !sr.RunContext.Config.InsecureSecrets {
logger.Debugf("Wrote command \n%s\n to '%s'", script, name) logger.Debugf("Wrote command \n%s\n to '%s'", script, name)
@@ -311,7 +317,30 @@ func (sr *stepRun) setupShellCommand(ctx context.Context) (name, script string,
rc := sr.getRunContext() rc := sr.getRunContext()
scriptPath := fmt.Sprintf("%s/%s", rc.JobContainer.GetActPath(), name) scriptPath := fmt.Sprintf("%s/%s", rc.JobContainer.GetActPath(), name)
if step.Shell == "pwsh" || step.Shell == "powershell" {
scriptPath = expressionStringEscape(scriptPath)
}
if step.Shell == "bash" || step.Shell == "sh" || step.Shell == "python" {
if rc.isWindowsHost() {
scriptPath = `"` + scriptPath + `"`
} else {
scriptPath = shellquote.Join(scriptPath)
}
}
sr.cmdline = strings.ReplaceAll(scCmd, `{0}`, scriptPath) sr.cmdline = strings.ReplaceAll(scCmd, `{0}`, scriptPath)
if rc.isWindowsHost() {
command := strings.TrimSpace(sr.cmdline)
if strings.HasPrefix(command, `"`) {
executable, _, closed := strings.Cut(command[1:], `"`)
if !closed || executable == "" {
return "", "", fmt.Errorf("invalid shell option %q: expected a double-quoted executable", step.Shell)
}
sr.cmd = []string{executable}
} else {
sr.cmd = strings.Fields(command)[:1]
}
return name, script, nil
}
sr.cmd, err = shellquote.Split(sr.cmdline) sr.cmd, err = shellquote.Split(sr.cmdline)
return name, script, err return name, script, err
+86 -23
View File
@@ -11,6 +11,7 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"runtime" "runtime"
"strings"
"testing" "testing"
"gitea.com/gitea/runner/act/container" "gitea.com/gitea/runner/act/container"
@@ -80,14 +81,17 @@ func TestStepRun(t *testing.T) {
} }
func TestStepRunShellParity(t *testing.T) { func TestStepRunShellParity(t *testing.T) {
tests := []struct { const powershellScript = "$ErrorActionPreference = 'stop'\n$OutputEncoding = [System.Text.UTF8Encoding]::new($false)\ntry { [Console]::OutputEncoding = $OutputEncoding } catch {}\necho 你好,世界\nif ((Test-Path -LiteralPath variable:/LASTEXITCODE)) { exit $LASTEXITCODE }"
for _, test := range []struct {
name, run, shell, workingDir string name, run, shell, workingDir string
hostOS, wantCmdline string
actPath string actPath string
env map[string]string env map[string]string
host bool host bool
probeErr error probeErr error
wantExt string wantExt string
wantCmd []string wantCmd []string
wantScript string
wantErr string wantErr string
}{ }{
{ {
@@ -105,24 +109,79 @@ func TestStepRunShellParity(t *testing.T) {
wantCmd: []string{"sh", "-e", "/var/run/act/workflow/1.sh"}, wantCmd: []string{"sh", "-e", "/var/run/act/workflow/1.sh"},
}, },
{ {
name: "builtin pwsh", name: "builtin pwsh",
shell: "pwsh", run: "echo 你好,世界",
actPath: "/var/run/act with spaces", shell: "pwsh",
wantExt: ".ps1", actPath: "/var/run/act with spaces/O'Brien",
wantCmd: []string{"pwsh", "-command", ". '/var/run/act with spaces/workflow/1.ps1'"}, wantExt: ".ps1",
wantCmd: []string{"pwsh", "-command", ". '/var/run/act with spaces/O''Brien/workflow/1.ps1'"},
wantScript: powershellScript,
}, },
{ {
name: "builtin powershell", name: "builtin powershell",
shell: "powershell", actPath: "/var/run/act/O'Brien",
wantExt: ".ps1", run: "echo 你好,世界",
wantCmd: []string{"powershell", "-command", ". '/var/run/act/workflow/1.ps1'"}, shell: "powershell",
wantExt: ".ps1",
wantCmd: []string{"powershell", "-command", ". '/var/run/act/O''Brien/workflow/1.ps1'"},
wantScript: "\ufeff" + powershellScript,
}, },
{ {
name: "custom pwsh template", name: "custom pwsh template",
shell: "pwsh -NoProfile -File {0}", run: "echo 你好,世界",
wantExt: ".ps1", shell: "pwsh -NoProfile -File {0}",
wantCmd: []string{"pwsh", "-NoProfile", "-File", "/var/run/act/workflow/1.ps1"}, wantExt: ".ps1",
wantCmd: []string{"pwsh", "-NoProfile", "-File", "/var/run/act/workflow/1.ps1"},
wantScript: powershellScript,
}, },
{
name: "builtin cmd reads and emits UTF-8 without a BOM",
run: "echo 你好,世界",
shell: "cmd",
wantExt: ".cmd",
wantCmd: []string{"cmd", "/D", "/E:ON", "/V:OFF", "/S", "/C", "CALL /var/run/act/workflow/1.cmd"},
wantScript: "@echo off\r\necho 你好,世界\r\n",
},
{
name: "Windows custom executable preserves backslashes and raw arguments",
shell: `C:\Python313\python.exe -X 'utf8' "{0}"`,
actPath: `C:\runner\O'Brien`,
hostOS: "windows",
wantCmd: []string{`C:\Python313\python.exe`},
wantCmdline: `C:\Python313\python.exe -X 'utf8' "{0}"`,
},
{
name: "Windows quoted custom executable preserves spaces and backslashes",
shell: `"C:\Program Files\Python313\python.exe" "{0}"`,
actPath: `C:\runner`,
hostOS: "windows",
wantCmd: []string{`C:\Program Files\Python313\python.exe`},
wantCmdline: `"C:\Program Files\Python313\python.exe" "{0}"`,
},
{
name: "Windows PowerShell escapes apostrophes in the raw command line",
run: "echo 你好,世界",
shell: "pwsh",
actPath: `C:\runner\O'Brien`,
hostOS: "windows",
wantExt: ".ps1",
wantCmd: []string{"pwsh"},
wantCmdline: `pwsh -command ". '{0}'"`,
wantScript: powershellScript,
},
{
name: "Windows unclosed executable quote",
shell: `"C:\Program Files\python.exe {0}`,
actPath: `C:\runner`,
hostOS: "windows",
wantErr: "expected a double-quoted executable",
},
{name: "linux builtin bash quotes script path", shell: "bash", actPath: `C:/Users/Jane Doe/O'Brien`, wantCmd: []string{"bash", "--noprofile", "--norc", "-e", "-o", "pipefail", `C:/Users/Jane Doe/O'Brien/workflow/1.sh`}},
{name: "linux builtin sh quotes script path", shell: "sh", actPath: `C:/Users/Jane Doe/O'Brien`, wantCmd: []string{"sh", "-e", `C:/Users/Jane Doe/O'Brien/workflow/1.sh`}},
{name: "linux builtin python quotes script path", shell: "python", actPath: `C:/Users/Jane Doe/O'Brien`, wantExt: ".py", wantCmd: []string{"python", `C:/Users/Jane Doe/O'Brien/workflow/1.py`}},
{name: "windows builtin bash quotes script path", shell: "bash", hostOS: "windows", actPath: `C:\Users\Jane Doe\O'Brien`, wantExt: ".sh", wantCmd: []string{"bash"}, wantCmdline: `bash --noprofile --norc -e -o pipefail "{0}"`},
{name: "windows builtin sh quotes script path", shell: "sh", hostOS: "windows", actPath: `C:\Users\Jane Doe\O'Brien`, wantExt: ".sh", wantCmd: []string{"sh"}, wantCmdline: `sh -e "{0}"`},
{name: "windows builtin python quotes script path", shell: "python", hostOS: "windows", actPath: `C:\Users\Jane Doe\O'Brien`, wantExt: ".py", wantCmd: []string{"python"}, wantCmdline: `python "{0}"`},
{ {
name: "missing placeholder", name: "missing placeholder",
shell: "bash -e", shell: "bash -e",
@@ -156,9 +215,7 @@ func TestStepRunShellParity(t *testing.T) {
workingDir: "${{ DIR }}", workingDir: "${{ DIR }}",
wantErr: "unable to interpolate the working directory:", wantErr: "unable to interpolate the working directory:",
}, },
} } {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) { t.Run(test.name, func(t *testing.T) {
cm := &containerMock{} cm := &containerMock{}
var jobContainer container.ExecutionsEnvironment = &shellContainerMock{cm} var jobContainer container.ExecutionsEnvironment = &shellContainerMock{cm}
@@ -189,22 +246,28 @@ func TestStepRunShellParity(t *testing.T) {
env: test.env, env: test.env,
} }
original := hostOS
hostOS = cmp.Or(test.hostOS, "linux")
t.Cleanup(func() { hostOS = original })
name, script, err := sr.setupShellCommand(t.Context()) name, script, err := sr.setupShellCommand(t.Context())
if test.wantErr != "" { if test.wantErr != "" {
require.ErrorContains(t, err, test.wantErr) require.ErrorContains(t, err, test.wantErr)
return return
} }
require.NoError(t, err) require.NoError(t, err)
if test.wantExt == "" { if test.wantExt == "" && test.hostOS != "windows" {
test.wantExt = ".sh" test.wantExt = ".sh"
} }
wantScript := "\necho hi\n"
if test.wantExt == ".ps1" {
wantScript = "$ErrorActionPreference = 'stop'\necho hi\nif ((Test-Path -LiteralPath variable:/LASTEXITCODE)) { exit $LASTEXITCODE }"
}
assert.Equal(t, "workflow/1"+test.wantExt, name) assert.Equal(t, "workflow/1"+test.wantExt, name)
assert.Equal(t, wantScript, script) assert.Equal(t, cmp.Or(test.wantScript, "\necho hi\n"), script)
assert.Equal(t, test.wantCmd, sr.cmd) assert.Equal(t, test.wantCmd, sr.cmd)
if test.wantCmdline != "" {
scriptPath := jobContainer.GetActPath() + "/workflow/1" + test.wantExt
if test.shell == "pwsh" || test.shell == "powershell" {
scriptPath = strings.ReplaceAll(scriptPath, "'", "''")
}
assert.Equal(t, strings.ReplaceAll(test.wantCmdline, "{0}", scriptPath), sr.cmdline)
}
assert.Equal(t, test.env["DIR"], sr.WorkingDirectory) assert.Equal(t, test.env["DIR"], sr.WorkingDirectory)
cm.AssertExpectations(t) cm.AssertExpectations(t)
}) })
+33 -1
View File
@@ -188,8 +188,9 @@ func TestSetupEnv(t *testing.T) {
"RC_KEY": "rcvalue", "RC_KEY": "rcvalue",
"RUNNER_ENVIRONMENT": "self-hosted", "RUNNER_ENVIRONMENT": "self-hosted",
"RUNNER_PERFLOG": "/dev/null", "RUNNER_PERFLOG": "/dev/null",
"RUNNER_TRACKING_ID": "", "RUNNER_TRACKING_ID": env["RUNNER_TRACKING_ID"],
} }
assert.Regexp(t, `^gitea_[A-Z2-7]{26}$`, env["RUNNER_TRACKING_ID"])
for name, value := range expected { for name, value := range expected {
if suffix, ok := strings.CutPrefix(name, "GITHUB_"); ok { if suffix, ok := strings.CutPrefix(name, "GITHUB_"); ok {
expected["GITEA_"+suffix] = value expected["GITEA_"+suffix] = value
@@ -197,6 +198,14 @@ func TestSetupEnv(t *testing.T) {
} }
assert.Equal(t, expected, env) assert.Equal(t, expected, env)
t.Run("keeps an inherited RUNNER_TRACKING_ID opt-out", func(t *testing.T) {
rc.Env["RUNNER_TRACKING_ID"] = ""
optedOut := map[string]string{}
require.NoError(t, setupEnv(t.Context(), &stepRun{RunContext: rc, Step: step, env: optedOut}))
assert.Contains(t, optedOut, "RUNNER_TRACKING_ID")
assert.Empty(t, optedOut["RUNNER_TRACKING_ID"])
})
for _, expression := range []string{"inputs.args", "env.ARGS"} { for _, expression := range []string{"inputs.args", "env.ARGS"} {
t.Run("deferred inputs from "+expression, func(t *testing.T) { t.Run("deferred inputs from "+expression, func(t *testing.T) {
workflow, err := model.ReadWorkflow(strings.NewReader(` workflow, err := model.ReadWorkflow(strings.NewReader(`
@@ -410,6 +419,29 @@ func (c *fileCommandTestContainer) GetContainerArchive(ctx context.Context, srcP
return c.HostEnvironment.GetContainerArchive(ctx, srcPath) return c.HostEnvironment.GetContainerArchive(ctx, srcPath)
} }
type windowsFileCommandTestContainer struct {
*container.HostEnvironment
}
func (*windowsFileCommandTestContainer) IsEnvironmentCaseInsensitive() bool { return true }
func TestProcessRunnerEnvFileCommandOrder(t *testing.T) {
rc := &RunContext{
JobContainer: &windowsFileCommandTestContainer{&container.HostEnvironment{ActPath: t.TempDir()}},
}
require.NoError(t, os.WriteFile(filepath.Join(rc.JobContainer.GetActPath(), "envs.txt"), []byte("BUILD_MODE=first\nbuild_mode<<EOF\nmiddle\nEOF\nBUILD_MODE=last\n"), 0o600))
var entries [][2]string
require.NoError(t, processRunnerEnvFileCommand(t.Context(), "envs.txt", rc, func(ctx context.Context, params map[string]string, value string) {
entries = append(entries, [2]string{params["name"], value})
rc.setEnvFile(ctx, params, value)
}))
assert.Equal(t, [][2]string{
{"BUILD_MODE", "first"}, {"build_mode", "middle"}, {"BUILD_MODE", "last"},
}, entries)
assert.Equal(t, map[string]string{"BUILD_MODE": "last"}, rc.Env)
assert.Equal(t, rc.Env, rc.GlobalEnv)
}
func TestRunStepExecutorParity(t *testing.T) { func TestRunStepExecutorParity(t *testing.T) {
newStep := func(t *testing.T, stepModel *model.Step) *stepRun { newStep := func(t *testing.T, stepModel *model.Step) *stepRun {
rc := createRunContext(t) rc := createRunContext(t)
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
echo "Unexpected value for `$env:key: $env:key" echo "Unexpected value for `$env:key: $env:key"
exit 1 exit 1
} }
if($env:key2 -ne "line1`nline2") { if($env:key2 -ne "line1`r`nline2") {
echo "Unexpected value for `$env:key2: $env:key2" echo "Unexpected value for `$env:key2: $env:key2"
exit 1 exit 1
} }
+14
View File
@@ -0,0 +1,14 @@
on: push
jobs:
test:
runs-on: windows-latest
steps:
- id: cmd
shell: cmd
run: echo 你好,世界
- id: pwsh
shell: pwsh
run: echo '你好,世界'
- id: powershell
shell: powershell
run: echo '你好,世界'
+18
View File
@@ -0,0 +1,18 @@
# Host-mode background processes on Windows
Windows host jobs clean up background step processes at job completion. A step can opt out:
```yaml
steps:
- name: Start shared helper
env:
RUNNER_TRACKING_ID: ""
run: Start-Process vendor-helper.exe
```
- Set the `RUNNER_TRACKING_ID` override in the step's `env` before it starts. Any value different from the job's ID opts out, including an empty string.
- Opted-out processes and descendants that inherit the override survive normal job cleanup, including workspace process sweeps and expired output waits. Detach helper output because the runner closes leftover output pipes after a bounded wait.
- Cancelling or timing out an active step still terminates its process tree.
- The override does not preserve workspace or temporary files. Store persistent helper files outside those directories and arrange the helper's eventual shutdown.
This follows [actions/runner's tracking ID and orphan cleanup](https://github.com/actions/runner/blob/main/src/Runner.Worker/JobExtension.cs). Container jobs follow their container's lifecycle.
+5 -1
View File
@@ -46,7 +46,11 @@ func Execute(ctx context.Context) {
Use: "daemon", Use: "daemon",
Short: "Run as a runner daemon", Short: "Run as a runner daemon",
Args: cobra.MaximumNArgs(0), Args: cobra.MaximumNArgs(0),
RunE: runDaemon(ctx, &daemArgs, &configFile), RunE: func(command *cobra.Command, args []string) error {
return runService(ctx, func(ctx context.Context) error {
return runDaemon(ctx, &daemArgs, &configFile)(command, args)
})
},
} }
daemonCmd.Flags().BoolVar(&daemArgs.Once, "once", false, "Run one job then exit") daemonCmd.Flags().BoolVar(&daemArgs.Once, "once", false, "Run one job then exit")
daemonCmd.Flags().StringVar(&daemArgs.Labels, "labels", os.Getenv("GITEA_RUNNER_LABELS"), "Runner labels, comma separated. Overrides the labels of an already registered runner") daemonCmd.Flags().StringVar(&daemArgs.Labels, "labels", os.Getenv("GITEA_RUNNER_LABELS"), "Runner labels, comma separated. Overrides the labels of an already registered runner")
+2 -1
View File
@@ -7,6 +7,7 @@ import (
"bytes" "bytes"
"os" "os"
"path/filepath" "path/filepath"
"strconv"
"testing" "testing"
"gitea.com/gitea/runner/internal/pkg/config" "gitea.com/gitea/runner/internal/pkg/config"
@@ -38,7 +39,7 @@ func TestConfigCmdInitWritesTheMinimalConfig(t *testing.T) {
out, _, err := runConfigCmd(t, file, "init") out, _, err := runConfigCmd(t, file, "init")
require.NoError(t, err) require.NoError(t, err)
assert.Contains(t, out, file) assert.Equal(t, "wrote config file "+strconv.Quote(file)+"\n", out)
content, err := os.ReadFile(file) content, err := os.ReadFile(file)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, config.Minimal, string(content)) assert.Equal(t, config.Minimal, string(content))
+1 -1
View File
@@ -145,7 +145,7 @@ func TestDaemonRetriesDeclareOutagesAndRemovesOnlyConsumedEphemeralRegistration(
{"ephemeral stopped during declare outage", true, true, connect.NewError(connect.CodeUnavailable, "bad gateway"), nil, false}, {"ephemeral stopped during declare outage", true, true, connect.NewError(connect.CodeUnavailable, "bad gateway"), nil, false},
} { } {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second) ctx, cancel := context.WithCancel(t.Context())
defer cancel() defer cancel()
service := &daemonTestService{declareErr: tc.declareErr, fetchErr: tc.fetchErr, beforeTask: tc.beforeTask, cancel: cancel} service := &daemonTestService{declareErr: tc.declareErr, fetchErr: tc.fetchErr, beforeTask: tc.beforeTask, cancel: cancel}
rpcServer := connect.NewServer() rpcServer := connect.NewServer()
+5 -2
View File
@@ -183,6 +183,9 @@ func TestRegisterInputsAssignToNext(t *testing.T) {
func TestInitInputs(t *testing.T) { func TestInitInputs(t *testing.T) {
t.Setenv(registerTokenEnvVar, "") t.Setenv(registerTokenEnvVar, "")
missingTokenFile := filepath.Join(t.TempDir(), "missing-token")
_, readErr := os.ReadFile(missingTokenFile)
require.ErrorIs(t, readErr, os.ErrNotExist)
for _, tc := range []struct { for _, tc := range []struct {
name, tokenFile, wantErr string name, tokenFile, wantErr string
env *string env *string
@@ -190,8 +193,8 @@ func TestInitInputs(t *testing.T) {
{name: "missing token", wantErr: "missing token, token-file argument, or GITEA_RUNNER_REGISTRATION_TOKEN environment variable"}, {name: "missing token", wantErr: "missing token, token-file argument, or GITEA_RUNNER_REGISTRATION_TOKEN environment variable"},
{name: "empty token", env: new(""), wantErr: "missing token, token-file argument, or GITEA_RUNNER_REGISTRATION_TOKEN environment variable"}, {name: "empty token", env: new(""), wantErr: "missing token, token-file argument, or GITEA_RUNNER_REGISTRATION_TOKEN environment variable"},
{ {
name: "invalid token file", tokenFile: "/tmp/nonexistent", env: new("from-env"), name: "invalid token file", tokenFile: missingTokenFile, env: new("from-env"),
wantErr: "cannot read the token file: /tmp/nonexistent, open /tmp/nonexistent: no such file or directory", wantErr: "cannot read the token file: " + missingTokenFile + ", " + readErr.Error(),
}, },
} { } {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
+12
View File
@@ -0,0 +1,12 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build !windows
package cmd
import "context"
func runService(ctx context.Context, run func(context.Context) error) error {
return run(ctx)
}
+107
View File
@@ -0,0 +1,107 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package cmd
import (
"context"
"errors"
"fmt"
"os"
"path/filepath"
"time"
"gitea.com/gitea/runner/act/common/logging"
"golang.org/x/sys/windows/svc"
)
func runService(ctx context.Context, run func(context.Context) error) error {
isService, err := svc.IsWindowsService()
if err != nil {
return fmt.Errorf("detect Windows service: %w", err)
}
if !isService {
return run(ctx)
}
handler := &daemonService{ctx: ctx, run: func(ctx context.Context) error {
executable, err := os.Executable()
if err != nil {
return err
}
if err := os.Chdir(filepath.Dir(executable)); err != nil {
return err
}
logFile, err := os.OpenFile("gitea-runner.log", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o600)
if err != nil {
return fmt.Errorf("open service log: %w", err)
}
defer func() { _ = logFile.Close() }()
previousOutput := logging.Output()
logging.SetOutput(logFile)
defer logging.SetOutput(previousOutput)
err = run(ctx)
if errors.Is(err, context.Canceled) && ctx.Err() != nil {
return nil
}
if err != nil {
logging.WithError(err).Error("runner daemon failed")
}
return err
}}
if err := svc.Run("", handler); err != nil { // SCM supplies the name for an own-process service.
return fmt.Errorf("run Windows service: %w", err)
}
return handler.err
}
type daemonService struct {
ctx context.Context
run func(context.Context) error
err error
}
func (s *daemonService) Execute(_args []string, requests <-chan svc.ChangeRequest, changes chan<- svc.Status) (bool, uint32) {
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
return s.execute(requests, changes, ticker.C)
}
func (s *daemonService) execute(requests <-chan svc.ChangeRequest, changes chan<- svc.Status, ticks <-chan time.Time) (bool, uint32) {
changes <- svc.Status{State: svc.StartPending}
ctx, cancel := context.WithCancel(s.ctx)
defer cancel()
done := make(chan error, 1)
go func() { done <- s.run(ctx) }()
status := svc.Status{State: svc.Running, Accepts: svc.AcceptStop | svc.AcceptShutdown | svc.AcceptPreShutdown}
changes <- status
for {
select {
case s.err = <-done:
if status.State != svc.StopPending {
changes <- svc.Status{State: svc.StopPending}
}
if s.err != nil {
return true, 1
}
return false, 0
case <-ticks:
if status.State == svc.StopPending {
status.CheckPoint++
changes <- status
}
case request := <-requests:
switch request.Cmd {
case svc.Interrogate:
changes <- status
case svc.Stop, svc.Shutdown, svc.PreShutdown:
if status.State != svc.StopPending {
status = svc.Status{State: svc.StopPending, CheckPoint: 1, WaitHint: 10000}
changes <- status
cancel()
}
}
}
}
}
+216
View File
@@ -0,0 +1,216 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package cmd
import (
"context"
"errors"
"fmt"
"os"
"path/filepath"
"testing"
"time"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/internal/pkg/config"
"github.com/stretchr/testify/require"
"golang.org/x/sys/windows"
"golang.org/x/sys/windows/svc"
"golang.org/x/sys/windows/svc/mgr"
)
type serviceExit struct {
specific bool
code uint32
}
func receiveServiceValue[T any](ctx context.Context, t *testing.T, values <-chan T) T {
t.Helper()
select {
case value := <-values:
return value
case <-ctx.Done():
t.Fatal(ctx.Err())
var zero T
return zero
}
}
func TestDaemonServiceControl(t *testing.T) {
for _, command := range []svc.Cmd{svc.Stop, svc.Shutdown, svc.PreShutdown} {
t.Run(fmt.Sprint(command), func(t *testing.T) {
ctx, cancel := context.WithTimeout(t.Context(), 10*time.Second)
defer cancel()
release, finish := context.WithCancel(ctx)
defer finish()
cancelled := make(chan struct{})
requests := make(chan svc.ChangeRequest, 1)
ticks := make(chan time.Time)
changes := make(chan svc.Status, 8)
done := make(chan serviceExit, 1)
handler := &daemonService{ctx: ctx, run: func(ctx context.Context) error {
<-ctx.Done()
close(cancelled)
<-release.Done()
return nil
}}
go func() {
specific, code := handler.execute(requests, changes, ticks)
done <- serviceExit{specific, code}
}()
require.Equal(t, svc.Status{State: svc.StartPending}, receiveServiceValue(ctx, t, changes))
running := svc.Status{State: svc.Running, Accepts: svc.AcceptStop | svc.AcceptShutdown | svc.AcceptPreShutdown}
require.Equal(t, running, receiveServiceValue(ctx, t, changes))
requests <- svc.ChangeRequest{Cmd: svc.Interrogate}
require.Equal(t, running, receiveServiceValue(ctx, t, changes))
requests <- svc.ChangeRequest{Cmd: command}
stopping := svc.Status{State: svc.StopPending, CheckPoint: 1, WaitHint: 10000}
require.Equal(t, stopping, receiveServiceValue(ctx, t, changes))
receiveServiceValue(ctx, t, cancelled)
for range 2 {
select {
case ticks <- time.Time{}:
case <-ctx.Done():
t.Fatal(ctx.Err())
}
stopping.CheckPoint++
require.Equal(t, stopping, receiveServiceValue(ctx, t, changes))
}
requests <- svc.ChangeRequest{Cmd: command}
requests <- svc.ChangeRequest{Cmd: svc.Interrogate}
require.Equal(t, stopping, receiveServiceValue(ctx, t, changes))
finish()
require.Equal(t, serviceExit{}, receiveServiceValue(ctx, t, done))
require.NoError(t, handler.err)
})
}
}
func TestDaemonServiceExit(t *testing.T) {
for _, daemonErr := range []error{nil, errors.New("daemon failed")} {
t.Run(fmt.Sprint(daemonErr), func(t *testing.T) {
ctx, cancel := context.WithTimeout(t.Context(), 10*time.Second)
defer cancel()
changes := make(chan svc.Status, 3)
done := make(chan serviceExit, 1)
handler := &daemonService{ctx: ctx, run: func(context.Context) error { return daemonErr }}
go func() {
specific, code := handler.Execute(nil, nil, changes)
done <- serviceExit{specific, code}
}()
require.Equal(t, svc.StartPending, receiveServiceValue(ctx, t, changes).State)
require.Equal(t, svc.Running, receiveServiceValue(ctx, t, changes).State)
require.Equal(t, svc.Status{State: svc.StopPending}, receiveServiceValue(ctx, t, changes))
if daemonErr == nil {
require.Equal(t, serviceExit{}, receiveServiceValue(ctx, t, done))
} else {
require.Equal(t, serviceExit{true, 1}, receiveServiceValue(ctx, t, done))
}
require.Equal(t, daemonErr, handler.err)
})
}
}
func waitForServiceState(t *testing.T, service *mgr.Service, state svc.State) svc.Status {
t.Helper()
ctx, cancel := context.WithTimeout(context.WithoutCancel(t.Context()), 20*time.Second)
defer cancel()
ticker := time.NewTicker(10 * time.Millisecond)
defer ticker.Stop()
for {
status, err := service.Query()
require.NoError(t, err)
if status.State == state {
return status
}
if state == svc.Running && status.State == svc.Stopped {
t.Fatalf("service stopped before running: %+v", status)
}
select {
case <-ticker.C:
case <-ctx.Done():
t.Fatalf("waiting for service state %d, last status: %+v", state, status)
}
}
}
func TestWindowsService(t *testing.T) {
isService, err := svc.IsWindowsService()
require.NoError(t, err)
if isService {
require.NoError(t, runService(t.Context(), func(ctx context.Context) error {
initLogging(&config.Config{})
logging.Info("service started")
<-ctx.Done()
logging.Info("service stopped")
return nil
}))
return
}
if !windows.GetCurrentProcessToken().IsElevated() {
t.Skip("Windows service test requires elevation")
}
executable, err := os.Executable()
require.NoError(t, err)
binary, err := os.ReadFile(executable)
require.NoError(t, err)
directory := t.TempDir()
serviceExecutable := filepath.Join(directory, "runner service.test.exe")
require.NoError(t, os.WriteFile(serviceExecutable, binary, 0o600))
manager, err := mgr.Connect()
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, manager.Disconnect()) })
service, err := manager.CreateService(
fmt.Sprintf("gitea-runner-test-%d-%d", os.Getpid(), time.Now().UnixNano()),
serviceExecutable, mgr.Config{StartType: mgr.StartManual}, "-test.run=^TestWindowsService$", "-test.timeout=1m",
)
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, service.Close()) })
t.Cleanup(func() { require.NoError(t, service.Delete()) })
var process windows.Handle
t.Cleanup(func() {
if process != 0 {
require.NoError(t, windows.CloseHandle(process))
}
})
t.Cleanup(func() {
if process == 0 {
return
}
result, err := windows.WaitForSingleObject(process, 20000)
require.NoError(t, err)
if result == uint32(windows.WAIT_TIMEOUT) {
t.Error("service process did not exit after stopping")
require.NoError(t, windows.TerminateProcess(process, 1))
result, err = windows.WaitForSingleObject(process, 20000)
require.NoError(t, err)
}
require.Equal(t, uint32(windows.WAIT_OBJECT_0), result)
})
t.Cleanup(func() {
status, err := service.Query()
require.NoError(t, err)
if status.State != svc.Stopped {
_, err := service.Control(svc.Stop)
require.NoError(t, err)
waitForServiceState(t, service, svc.Stopped)
}
})
require.NoError(t, service.Start())
running := waitForServiceState(t, service, svc.Running)
process, err = windows.OpenProcess(windows.SYNCHRONIZE|windows.PROCESS_TERMINATE, false, running.ProcessId)
require.NoError(t, err)
_, err = service.Control(svc.Stop)
require.NoError(t, err)
stopped := waitForServiceState(t, service, svc.Stopped)
require.Zero(t, stopped.Win32ExitCode)
require.Zero(t, stopped.ServiceSpecificExitCode)
log, err := os.ReadFile(filepath.Join(directory, "gitea-runner.log"))
require.NoError(t, err)
require.Contains(t, string(log), "service started")
require.Contains(t, string(log), "service stopped")
}
+1 -2
View File
@@ -59,9 +59,8 @@ func (r *Runner) checkConfiguredHealth(ctx context.Context) (bool, string) {
func executeHealthCheck(ctx context.Context, script string, timeout time.Duration, env map[string]string) error { func executeHealthCheck(ctx context.Context, script string, timeout time.Duration, env map[string]string) error {
checkCtx, cancel := context.WithTimeout(ctx, timeout) checkCtx, cancel := context.WithTimeout(ctx, timeout)
defer cancel() defer cancel()
cmd := exec.CommandContext(checkCtx, script) cmd := scriptCommand(checkCtx, script)
cmd.Env = envListFromMap(env) cmd.Env = envListFromMap(env)
cmd.SysProcAttr = process.SysProcAttr(script, false)
writer := common.NewLineWriter(func(line string) bool { writer := common.NewLineWriter(func(line string) bool {
line = strings.TrimRight(line, "\r\n") line = strings.TrimRight(line, "\r\n")
if line != "" { if line != "" {
@@ -0,0 +1,25 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package run
import (
"os"
"path/filepath"
"testing"
"time"
"github.com/stretchr/testify/require"
)
func TestExecuteHealthCheckExeAndBatchPathsWithSpace(t *testing.T) {
whoami, err := os.ReadFile(filepath.Join(os.Getenv("SystemRoot"), "System32", "whoami.exe"))
require.NoError(t, err)
dir := filepath.Join(t.TempDir(), "health check")
require.NoError(t, os.MkdirAll(dir, 0o755))
require.NoError(t, os.WriteFile(filepath.Join(dir, "probe.exe"), whoami, 0o755))
require.NoError(t, os.WriteFile(filepath.Join(dir, "probe.cmd"), []byte("@exit /b 0\r\n"), 0o755))
for _, name := range []string{"probe.exe", "probe.cmd"} {
require.NoError(t, executeHealthCheck(t.Context(), filepath.Join(dir, name), time.Minute, nil))
}
}
+14 -2
View File
@@ -8,6 +8,8 @@ import (
"errors" "errors"
"io" "io"
"os/exec" "os/exec"
"path/filepath"
"runtime"
"strconv" "strconv"
"strings" "strings"
"time" "time"
@@ -39,9 +41,8 @@ func (r *Runner) runPostTaskScript(ctx context.Context, reporter *report.Reporte
env := r.postTaskScriptEnv(reporter, task, workdir) env := r.postTaskScriptEnv(reporter, task, workdir)
logging.Infof("running post-task script %q for task %d", script, task.Id) logging.Infof("running post-task script %q for task %d", script, task.Id)
cmd := exec.CommandContext(scriptCtx, script) cmd := scriptCommand(scriptCtx, script)
cmd.Env = envListFromMap(env) cmd.Env = envListFromMap(env)
cmd.SysProcAttr = process.SysProcAttr(script, false)
stdout := postTaskScriptLogWriter("stdout") stdout := postTaskScriptLogWriter("stdout")
stderr := postTaskScriptLogWriter("stderr") stderr := postTaskScriptLogWriter("stderr")
@@ -75,6 +76,17 @@ func (r *Runner) runPostTaskScript(ctx context.Context, reporter *report.Reporte
} }
} }
func scriptCommand(ctx context.Context, script string) *exec.Cmd { // Windows batch files need cmd.exe, CreateProcess cannot start them
if ext := strings.ToLower(filepath.Ext(script)); runtime.GOOS == "windows" && (ext == ".bat" || ext == ".cmd") {
cmd := exec.CommandContext(ctx, "cmd.exe")
cmd.SysProcAttr = process.SysProcAttr(`cmd.exe /d /s /c ""`+script+`""`, false)
return cmd
}
cmd := exec.CommandContext(ctx, script)
cmd.SysProcAttr = process.SysProcAttr("", false)
return cmd
}
func postTaskScriptContext(ctx context.Context, timeout time.Duration) (context.Context, context.CancelFunc) { func postTaskScriptContext(ctx context.Context, timeout time.Duration) (context.Context, context.CancelFunc) {
// Detach from the task context's deadline and cancellation: the task has // Detach from the task context's deadline and cancellation: the task has
// already finished by the time the post-task script runs, so the script must // already finished by the time the post-task script runs, so the script must
+17 -2
View File
@@ -7,10 +7,12 @@ import (
"context" "context"
"os" "os"
"path/filepath" "path/filepath"
"runtime"
"testing" "testing"
"time" "time"
"gitea.com/gitea/runner/act/common/logging" "gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
"gitea.com/gitea/runner/internal/pkg/config" "gitea.com/gitea/runner/internal/pkg/config"
"gitea.com/gitea/runner/internal/pkg/metrics" "gitea.com/gitea/runner/internal/pkg/metrics"
"gitea.com/gitea/runner/internal/pkg/report" "gitea.com/gitea/runner/internal/pkg/report"
@@ -39,9 +41,15 @@ func TestRunPostTaskScriptSkippedWhenEmpty(t *testing.T) {
} }
func TestRunPostTaskScriptNonZeroExitDoesNotPanic(t *testing.T) { func TestRunPostTaskScriptNonZeroExitDoesNotPanic(t *testing.T) {
hook := logtest.SetDefault(t)
dir := t.TempDir() dir := t.TempDir()
scriptPath := filepath.Join(dir, "fail.sh") scriptPath := filepath.Join(dir, "fail.sh")
require.NoError(t, os.WriteFile(scriptPath, []byte("#!/bin/sh\nexit 2\n"), 0o700)) script := "#!/bin/sh\nexit 2\n"
if runtime.GOOS == "windows" {
scriptPath = filepath.Join(dir, "fail.cmd")
script = "@exit /b 2\r\n"
}
require.NoError(t, os.WriteFile(scriptPath, []byte(script), 0o700))
cfg, err := config.LoadDefault("") cfg, err := config.LoadDefault("")
require.NoError(t, err) require.NoError(t, err)
@@ -59,6 +67,8 @@ func TestRunPostTaskScriptNonZeroExitDoesNotPanic(t *testing.T) {
require.NotPanics(t, func() { require.NotPanics(t, func() {
r.runPostTaskScript(ctx, reporter, task, "/workspace/owner/repo") r.runPostTaskScript(ctx, reporter, task, "/workspace/owner/repo")
}) })
require.NotNil(t, hook.LastRecord())
assert.Contains(t, hook.LastRecord().Message, "exited with code 2")
} }
func TestPostTaskScriptContextUsesFullTimeout(t *testing.T) { func TestPostTaskScriptContextUsesFullTimeout(t *testing.T) {
@@ -113,10 +123,15 @@ func TestPostTaskScriptEnv(t *testing.T) {
} }
func TestRunPostTaskScriptIntegration(t *testing.T) { func TestRunPostTaskScriptIntegration(t *testing.T) {
dir := t.TempDir() dir := filepath.Join(t.TempDir(), "script directory")
require.NoError(t, os.Mkdir(dir, 0o700))
outFile := filepath.Join(dir, "out.txt") outFile := filepath.Join(dir, "out.txt")
scriptPath := filepath.Join(dir, "post-task.sh") scriptPath := filepath.Join(dir, "post-task.sh")
script := "#!/bin/sh\nprintf '%s %s %s' \"$GITEA_TASK_ID\" \"$GITEA_JOB_RESULT\" \"$CUSTOM\" > \"" + outFile + "\"\n" script := "#!/bin/sh\nprintf '%s %s %s' \"$GITEA_TASK_ID\" \"$GITEA_JOB_RESULT\" \"$CUSTOM\" > \"" + outFile + "\"\n"
if runtime.GOOS == "windows" {
scriptPath = filepath.Join(dir, "post-task.cmd")
script = "@echo off\r\n<nul set /p \"=%GITEA_TASK_ID% %GITEA_JOB_RESULT% %CUSTOM%\" > \"" + outFile + "\"\r\nexit /b 0\r\n"
}
require.NoError(t, os.WriteFile(scriptPath, []byte(script), 0o700)) require.NoError(t, os.WriteFile(scriptPath, []byte(script), 0o700))
cfg, err := config.LoadDefault("") cfg, err := config.LoadDefault("")
+14 -9
View File
@@ -168,7 +168,7 @@ func (r *Runner) OnIdle(ctx context.Context) {
} }
// Bind-workdir mode: reclaim stale per-task workspace dirs (numeric task IDs). // Bind-workdir mode: reclaim stale per-task workspace dirs (numeric task IDs).
if r.cfg.Container.BindWorkdir { if r.cfg.Container.BindWorkdir {
r.cleanupStaleDirs(ctx, filepath.FromSlash("/"+strings.TrimLeft(r.cfg.Container.WorkdirParent, "/")), isTaskIDDir) r.cleanupStaleDirs(ctx, containerWorkdirRoot(r.cfg.Container.WorkdirParent), isTaskIDDir)
} }
// Host mode: reclaim per-job scratch dirs left behind when HostEnvironment // Host mode: reclaim per-job scratch dirs left behind when HostEnvironment
// cleanup timed out (e.g. a delete stalled by an AV/EDR filter driver). They // cleanup timed out (e.g. a delete stalled by an AV/EDR filter driver). They
@@ -557,12 +557,12 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
actionCloneDepth = 0 actionCloneDepth = 0
} }
workdirParent := strings.TrimLeft(r.cfg.Container.WorkdirParent, "/") workdirParent := containerWorkdirRoot(r.cfg.Container.WorkdirParent)
if r.cfg.Container.BindWorkdir { if r.cfg.Container.BindWorkdir {
// Append the task ID to isolate concurrent jobs from the same repo. // Append the task ID to isolate concurrent jobs from the same repo.
workdirParent = fmt.Sprintf("%s/%d", workdirParent, task.Id) workdirParent = filepath.Join(workdirParent, strconv.FormatInt(task.Id, 10))
} }
workdir := filepath.FromSlash(fmt.Sprintf("/%s/%s", workdirParent, preset.Repository)) workdir := filepath.Join(workdirParent, filepath.FromSlash(preset.Repository))
if runtime.GOOS == "windows" { if runtime.GOOS == "windows" {
if abs, err := filepath.Abs(workdir); err == nil { if abs, err := filepath.Abs(workdir); err == nil {
workdir = abs workdir = abs
@@ -648,10 +648,8 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
reporter.SetOutputs(job.Outputs) reporter.SetOutputs(job.Outputs)
if r.cfg.Container.BindWorkdir { if r.cfg.Container.BindWorkdir {
// Remove the entire task-specific directory (e.g. /workspace/<task_id>). if err := os.RemoveAll(workdirParent); err != nil {
taskDir := filepath.FromSlash("/" + workdirParent) logging.Warnf("failed to clean up workspace %s: %v", workdirParent, err)
if err := os.RemoveAll(taskDir); err != nil {
logging.Warnf("failed to clean up workspace %s: %v", taskDir, err)
} }
} }
@@ -821,7 +819,7 @@ func (r *Runner) setHealthStatus(ready bool, reason string) {
func checkFreeDisk(cfg *config.Config) (bool, string) { func checkFreeDisk(cfg *config.Config) (bool, string) {
root := cfg.Host.WorkdirParent root := cfg.Host.WorkdirParent
if cfg.Container.BindWorkdir { if cfg.Container.BindWorkdir {
root = filepath.FromSlash("/" + strings.TrimLeft(cfg.Container.WorkdirParent, "/")) root = containerWorkdirRoot(cfg.Container.WorkdirParent)
} }
root = nearestExistingPath(root) root = nearestExistingPath(root)
available, err := disk.FreeBytes(root) available, err := disk.FreeBytes(root)
@@ -835,6 +833,13 @@ func checkFreeDisk(cfg *config.Config) (bool, string) {
return true, "ok" return true, "ok"
} }
func containerWorkdirRoot(parent string) string {
if filepath.IsAbs(parent) {
return filepath.Clean(parent)
}
return filepath.Join(string(filepath.Separator), parent)
}
func nearestExistingPath(path string) string { func nearestExistingPath(path string) string {
for path != "" { for path != "" {
if _, err := os.Stat(path); err == nil { if _, err := os.Stat(path); err == nil {
+14 -4
View File
@@ -241,9 +241,6 @@ func TestRunnerShouldRunIdleCleanupSkipsWhenDisabled(t *testing.T) {
}) })
} }
// TestRunnerOnIdleIntegratesCleanup wires the full OnIdle entry point and
// confirms it walks workdir_parent (after the leading-slash trim that
// matches the production path construction) and removes stale numeric dirs.
func TestRunnerOnIdleIntegratesCleanup(t *testing.T) { func TestRunnerOnIdleIntegratesCleanup(t *testing.T) {
now := time.Date(2026, time.April, 29, 20, 0, 0, 0, time.UTC) now := time.Date(2026, time.April, 29, 20, 0, 0, 0, time.UTC)
root := t.TempDir() root := t.TempDir()
@@ -255,7 +252,7 @@ func TestRunnerOnIdleIntegratesCleanup(t *testing.T) {
cfg: &config.Config{ cfg: &config.Config{
Container: config.Container{ Container: config.Container{
BindWorkdir: true, BindWorkdir: true,
WorkdirParent: root, // leading slash absent, OnIdle reattaches it WorkdirParent: root,
}, },
Runner: config.Runner{ Runner: config.Runner{
WorkdirCleanupAge: 24 * time.Hour, WorkdirCleanupAge: 24 * time.Hour,
@@ -371,3 +368,16 @@ func TestRunnerOnIdleRemovesOrphanNetworks(t *testing.T) {
assert.Equal(t, []string{"a1b2c3d4e5f6", ""}, detached) assert.Equal(t, []string{"a1b2c3d4e5f6", ""}, detached)
assert.Equal(t, []string{"runner-1"}, sweptPods) assert.Equal(t, []string{"runner-1"}, sweptPods)
} }
func TestContainerWorkdirRoot(t *testing.T) {
absolute := t.TempDir()
rooted := filepath.Join(string(filepath.Separator), "workspace")
for input, want := range map[string]string{
"workspace": rooted,
"/workspace/": rooted,
absolute: absolute,
absolute + "/nested": filepath.Join(absolute, "nested"),
} {
assert.Equal(t, want, containerWorkdirRoot(input), input)
}
}
+25 -14
View File
@@ -42,7 +42,10 @@ func Main(ctx context.Context, c *action.Context) (err error) {
return fmt.Errorf("checkout: unsupported input %q", name) return fmt.Errorf("checkout: unsupported input %q", name)
} }
} }
dst, err := workspaceDestination(c.Workspace, c.Inputs["path"]) runnerContext := c.Container.GetRunnerContext(ctx)
executionOS, _ := runnerContext["os"].(string)
windows := strings.EqualFold(executionOS, "windows")
dst, err := workspaceDestination(c.Workspace, c.Inputs["path"], windows)
if err != nil { if err != nil {
return err return err
} }
@@ -63,8 +66,8 @@ func Main(ctx context.Context, c *action.Context) (err error) {
return err return err
} }
} }
if output := c.Env["GITHUB_OUTPUT"]; output != "" { if output := filepath.ToSlash(c.Env["GITHUB_OUTPUT"]); output != "" {
return c.Container.Copy(path.Dir(output), &container.FileEntry{Name: path.Base(output), Mode: 0o666, Body: fmt.Sprintf("ref=%s\ncommit=%s\n", ghc.Ref, ghc.Sha)})(ctx) return c.Container.Copy(container.JoinPath(output, ".."), &container.FileEntry{Name: path.Base(output), Mode: 0o666, Body: fmt.Sprintf("ref=%s\ncommit=%s\n", ghc.Ref, ghc.Sha)})(ctx)
} }
return nil return nil
} }
@@ -72,6 +75,7 @@ func Main(ctx context.Context, c *action.Context) (err error) {
co := &checkout{ co := &checkout{
url: strings.TrimSuffix(ghc.ServerURL, "/") + "/" + repository, url: strings.TrimSuffix(ghc.ServerURL, "/") + "/" + repository,
dst: dst, dst: dst,
windows: windows,
ref: ref, ref: ref,
depth: depth, depth: depth,
fetchTags: boolInput(c.Inputs, "fetch-tags", false), fetchTags: boolInput(c.Inputs, "fetch-tags", false),
@@ -114,10 +118,11 @@ func Main(ctx context.Context, c *action.Context) (err error) {
} }
// named like actions/checkout's, which create-pull-request knows to hide, and gone with the job // named like actions/checkout's, which create-pull-request knows to hide, and gone with the job
runnerTemp, _ := c.Container.GetRunnerContext(ctx)["temp"].(string) runnerTemp, _ := runnerContext["temp"].(string)
runnerTemp = filepath.ToSlash(runnerTemp)
digest := sha256.Sum256([]byte(dst)) digest := sha256.Sum256([]byte(dst))
credentials := fmt.Sprintf("git-credentials-%x.config", digest[:8]) credentials := fmt.Sprintf("git-credentials-%x.config", digest[:8])
co.credentials = path.Join(runnerTemp, credentials) co.credentials = container.JoinPath(runnerTemp, credentials)
persist := boolInput(c.Inputs, "persist-credentials", true) persist := boolInput(c.Inputs, "persist-credentials", true)
config, remoteConfig := serverConfig(ghc.ServerURL, cmp.Or(c.Inputs["token"], ghc.Token), c.InsecureSkipTLS, co.submodules != "" && sshKey == "") config, remoteConfig := serverConfig(ghc.ServerURL, cmp.Or(c.Inputs["token"], ghc.Token), c.InsecureSkipTLS, co.submodules != "" && sshKey == "")
if boolInput(c.Inputs, "set-safe-directory", true) { if boolInput(c.Inputs, "set-safe-directory", true) {
@@ -140,22 +145,21 @@ func Main(ctx context.Context, c *action.Context) (err error) {
if err := c.Container.Copy(runnerTemp, &container.FileEntry{Name: cert, Mode: 0o600, Body: string(certPEM)}, &container.FileEntry{Name: key, Mode: 0o600, Body: string(keyPEM)})(ctx); err != nil { if err := c.Container.Copy(runnerTemp, &container.FileEntry{Name: cert, Mode: 0o600, Body: string(certPEM)}, &container.FileEntry{Name: key, Mode: 0o600, Body: string(keyPEM)})(ctx); err != nil {
return err return err
} }
scope := "http." + strings.TrimSuffix(ghc.ServerURL, "/") + "/." remoteConfig = append(remoteConfig, git.ClientCertConfig("http."+strings.TrimSuffix(ghc.ServerURL, "/")+"/.", container.JoinPath(runnerTemp, cert), container.JoinPath(runnerTemp, key), executionOS)...)
remoteConfig = append(remoteConfig, scope+"sslCert="+path.Join(runnerTemp, cert), scope+"sslKey="+path.Join(runnerTemp, key))
} }
env := gitEnv(c, remoteConfig) env := gitEnv(c, remoteConfig)
env["GIT_LFS_SKIP_SMUDGE"] = strconv.FormatBool(!co.lfs || len(co.sparse) == 0) // lfs pull fetches the objects of a full checkout in one go env["GIT_LFS_SKIP_SMUDGE"] = strconv.FormatBool(!co.lfs || len(co.sparse) == 0) // lfs pull fetches the objects of a full checkout in one go
if sshKey != "" { if sshKey != "" {
key := fmt.Sprintf("git-ssh-key-%x", digest[:8]) key := fmt.Sprintf("git-ssh-key-%x", digest[:8])
files := []*container.FileEntry{{Name: key, Mode: 0o600, Body: sshKey + "\n"}} files := []*container.FileEntry{{Name: key, Mode: 0o600, Body: sshKey + "\n"}}
ssh := []string{"ssh", "-i", path.Join(runnerTemp, key)} ssh := []string{"ssh", "-i", container.JoinPath(runnerTemp, key)}
if boolInput(c.Inputs, "ssh-strict", true) { if boolInput(c.Inputs, "ssh-strict", true) {
ssh = append(ssh, "-o", "StrictHostKeyChecking=yes", "-o", "CheckHostIP=no") ssh = append(ssh, "-o", "StrictHostKeyChecking=yes", "-o", "CheckHostIP=no")
} }
if knownHosts := strings.TrimSpace(c.Inputs["ssh-known-hosts"]); knownHosts != "" { if knownHosts := strings.TrimSpace(c.Inputs["ssh-known-hosts"]); knownHosts != "" {
name := fmt.Sprintf("git-ssh-known-hosts-%x", digest[:8]) name := fmt.Sprintf("git-ssh-known-hosts-%x", digest[:8])
files = append(files, &container.FileEntry{Name: name, Mode: 0o600, Body: knownHosts + "\n"}) files = append(files, &container.FileEntry{Name: name, Mode: 0o600, Body: knownHosts + "\n"})
ssh = append(ssh, "-o", "UserKnownHostsFile=~/.ssh/known_hosts "+strconv.Quote(path.Join(runnerTemp, name))) ssh = append(ssh, "-o", "UserKnownHostsFile=~/.ssh/known_hosts "+strconv.Quote(container.JoinPath(runnerTemp, name)))
} }
if !persist { if !persist {
defer func() { defer func() {
@@ -218,7 +222,7 @@ func gitEnv(c *action.Context, config []string) map[string]string {
env := map[string]string{"GIT_TERMINAL_PROMPT": "0"} env := map[string]string{"GIT_TERMINAL_PROMPT": "0"}
maps.Copy(env, c.Env) maps.Copy(env, c.Env)
maps.DeleteFunc(env, func(name, _ string) bool { return slices.Contains(git.LocalEnvVars, name) }) maps.DeleteFunc(env, func(name, _ string) bool { return slices.Contains(git.LocalEnvVars, name) })
maps.Copy(env, git.ConfigEnv(append([]string{"protocol.version=2", "init.defaultBranch=main", "advice.detachedHead=false", "maintenance.auto=false"}, config...)...)) maps.Copy(env, git.ConfigEnv(append([]string{"protocol.version=2", "init.defaultBranch=main", "advice.detachedHead=false", "maintenance.auto=false", "core.longpaths=true"}, config...)...))
return env return env
} }
@@ -301,7 +305,7 @@ type checkout struct {
filter, submodules string filter, submodules string
sparse []string sparse []string
depth int depth int
fetchTags, clean, lfs, progress, cone bool fetchTags, clean, lfs, progress, cone, windows bool
} }
const fullDepth = "2147483647" // git's INFINITE_DEPTH, also deepens a shallow dst const fullDepth = "2147483647" // git's INFINITE_DEPTH, also deepens a shallow dst
@@ -315,6 +319,9 @@ func (co *checkout) commands() (commands [][]string, fetched int) {
repo("config", "remote.origin.fetch", "+refs/heads/*:refs/remotes/origin/*"), repo("config", "remote.origin.fetch", "+refs/heads/*:refs/remotes/origin/*"),
repo("config", "--replace-all", "--fixed-value", credentialsInclude, co.credentials, co.credentials), repo("config", "--replace-all", "--fixed-value", credentialsInclude, co.credentials, co.credentials),
} }
if co.windows {
commands = append(commands, repo("config", "core.longpaths", "true")) // later steps run git on the checkout too
}
if co.lfs { if co.lfs {
commands = append(commands, repo("lfs", "install", "--local")) commands = append(commands, repo("lfs", "install", "--local"))
} }
@@ -451,9 +458,13 @@ func boolInput(inputs map[string]string, name string, fallback bool) bool {
return strings.EqualFold(value, "true") || value == "" && fallback return strings.EqualFold(value, "true") || value == "" && fallback
} }
func workspaceDestination(workspace, checkoutPath string) (string, error) { func workspaceDestination(workspace, checkoutPath string, windows bool) (string, error) {
cleanPath := path.Clean(strings.TrimSpace(checkoutPath)) cleanPath := strings.TrimSpace(checkoutPath)
if path.IsAbs(cleanPath) || cleanPath == ".." || strings.HasPrefix(cleanPath, "../") { if windows {
cleanPath = strings.ReplaceAll(cleanPath, `\`, "/")
}
cleanPath = path.Clean(cleanPath)
if path.IsAbs(cleanPath) || cleanPath == ".." || strings.HasPrefix(cleanPath, "../") || windows && len(cleanPath) > 1 && cleanPath[1] == ':' {
return "", fmt.Errorf("checkout: path %q must be relative to the workspace", checkoutPath) return "", fmt.Errorf("checkout: path %q must be relative to the workspace", checkoutPath)
} }
return path.Join(workspace, cleanPath), nil return path.Join(workspace, cleanPath), nil
+31 -3
View File
@@ -13,9 +13,11 @@ import (
"io" "io"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/url"
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"runtime"
"strings" "strings"
"testing" "testing"
@@ -50,6 +52,7 @@ func gitLog(dir string) []string {
func TestCheckout(t *testing.T) { func TestCheckout(t *testing.T) {
server := t.TempDir() server := t.TempDir()
serverURL := (&url.URL{Scheme: "file", Path: "/" + strings.TrimPrefix(filepath.ToSlash(server), "/")}).String()
var sha string var sha string
_, lfsErr := exec.LookPath("git-lfs") _, lfsErr := exec.LookPath("git-lfs")
require.True(t, t.Run("fixtures", func(t *testing.T) { require.True(t, t.Run("fixtures", func(t *testing.T) {
@@ -192,13 +195,16 @@ exec sh -c "$(printf '%s' "$last" | sed "s#git-upload-pack '/\(.*\)\.git'#git up
if tc.inputs["lfs"] == "true" && lfsErr != nil { if tc.inputs["lfs"] == "true" && lfsErr != nil {
t.Skip("needs git-lfs") t.Skip("needs git-lfs")
} }
if tc.inputs["ssh-key"] != "" && runtime.GOOS == "windows" {
t.Skip("Git for Windows puts its own ssh ahead of the fake one on PATH")
}
workspace, runnerTemp := t.TempDir(), t.TempDir() workspace, runnerTemp := t.TempDir(), t.TempDir()
output := filepath.Join(runnerTemp, "output") output := filepath.Join(runnerTemp, "output")
checkout := func(inputs map[string]string) error { checkout := func(inputs map[string]string) error {
return Main(t.Context(), &action.Context{ return Main(t.Context(), &action.Context{
Container: &container.HostEnvironment{Path: workspace, TmpDir: runnerTemp, StdOut: io.Discard}, Container: &container.HostEnvironment{Path: workspace, TmpDir: runnerTemp, StdOut: io.Discard},
Github: &model.GithubContext{ Github: &model.GithubContext{
Repository: "org/repo", Ref: cmp.Or(tc.eventRef, "refs/heads/main"), Sha: sha, ServerURL: "file://" + server, Token: "tok", Event: tc.event, Repository: "org/repo", Ref: cmp.Or(tc.eventRef, "refs/heads/main"), Sha: sha, ServerURL: serverURL, Token: "tok", Event: tc.event,
}, },
Inputs: inputs, Inputs: inputs,
HostWorkdir: host, HostWorkdir: host,
@@ -207,7 +213,7 @@ exec sh -c "$(printf '%s' "$last" | sed "s#git-upload-pack '/\(.*\)\.git'#git up
NoSkipCheckout: tc.noSkip, NoSkipCheckout: tc.noSkip,
Env: map[string]string{ Env: map[string]string{
"PATH": tools + string(os.PathListSeparator) + os.Getenv("PATH"), "GIT_CONFIG_GLOBAL": globalConfig, "GIT_CONFIG_NOSYSTEM": "1", "PATH": tools + string(os.PathListSeparator) + os.Getenv("PATH"), "GIT_CONFIG_GLOBAL": globalConfig, "GIT_CONFIG_NOSYSTEM": "1",
"GIT_DIR": host, "GITHUB_OUTPUT": output, "SSH_SERVER": server, "SSH_ARGS": filepath.Join(runnerTemp, "ssh-args"), "GIT_DIR": host, "GITHUB_OUTPUT": output, "SSH_SERVER": filepath.ToSlash(server), "SSH_ARGS": filepath.Join(runnerTemp, "ssh-args"),
}, },
}) })
} }
@@ -270,7 +276,7 @@ exec sh -c "$(printf '%s' "$last" | sed "s#git-upload-pack '/\(.*\)\.git'#git up
for _, repo := range repos { for _, repo := range repos {
assert.Equal(t, tc.inputs["clean"] == "false", fileExists(filepath.Join(repo, "untracked")), repo) assert.Equal(t, tc.inputs["clean"] == "false", fileExists(filepath.Join(repo, "untracked")), repo)
assert.Len(t, strings.Fields(gitOutput(repo, "config", "--local", "--get-all", credentialsInclude)), 1, repo) assert.Len(t, strings.Fields(gitOutput(repo, "config", "--local", "--get-all", credentialsInclude)), 1, repo)
headers, _ := exec.Command("git", "-C", repo, "config", "--get-all", "http.file://"+server+"/.extraheader").Output() headers, _ := exec.Command("git", "-C", repo, "config", "--get-all", "http."+serverURL+"/.extraheader").Output()
assert.Equal(t, persisted, string(headers) == "\nAuthorization: Basic dG9rZW46dG9r\n", repo) assert.Equal(t, persisted, string(headers) == "\nAuthorization: Basic dG9rZW46dG9r\n", repo)
} }
if tc.inputs["ssh-key"] != "" { if tc.inputs["ssh-key"] != "" {
@@ -354,3 +360,25 @@ func fileExists(name string) bool {
_, err := os.Stat(name) _, err := os.Stat(name)
return err == nil return err == nil
} }
func TestWorkspaceDestination(t *testing.T) {
for _, tc := range []struct {
path string
windows bool
want string
}{
{`..\x`, true, ""},
{`C:x`, true, ""},
{`a\b`, true, "/w/a/b"},
{`..\x`, false, `/w/..\x`},
{`a:b`, false, "/w/a:b"},
} {
got, err := workspaceDestination("/w", tc.path, tc.windows)
if tc.want == "" {
assert.ErrorContains(t, err, "must be relative", tc.path)
} else {
require.NoError(t, err, tc.path)
assert.Equal(t, tc.want, got, tc.path)
}
}
}
+2 -2
View File
@@ -327,8 +327,8 @@ health_check:
# Minimum free space required on the filesystem holding runner workspaces. # Minimum free space required on the filesystem holding runner workspaces.
# Defaults to 1024 MiB when omitted or set to zero. # Defaults to 1024 MiB when omitted or set to zero.
#min_free_disk_space_mb: 1024 #min_free_disk_space_mb: 1024
# Optional additional executable. A non-zero exit, timeout, or startup failure # Optional additional executable, or a .bat or .cmd file on Windows. A non-zero
# marks the runner unavailable. # exit, timeout, or startup failure marks the runner unavailable.
#script: '' #script: ''
# How long a script result is cached and its maximum execution time. # How long a script result is cached and its maximum execution time.
#interval: 30s #interval: 30s
+6 -6
View File
@@ -109,7 +109,7 @@ cache:
endpoint: "http://minio:9000" endpoint: "http://minio:9000"
bucket: "caches" bucket: "caches"
prefix: "/runners/" prefix: "/runners/"
access_key_file: "` + accessPath + `" access_key_file: '` + accessPath + `'
secret_key: "secret" secret_key: "secret"
` `
require.NoError(t, os.WriteFile(path, []byte(content), 0o600)) require.NoError(t, os.WriteFile(path, []byte(content), 0o600))
@@ -341,20 +341,20 @@ func TestLoadDefault_ExternalCacheSecret(t *testing.T) {
{name: "without secret", yaml: externalCache, wantErr: "external_secret"}, {name: "without secret", yaml: externalCache, wantErr: "external_secret"},
{name: "inline secret", yaml: externalCache + " external_secret: \"shh\"\n"}, {name: "inline secret", yaml: externalCache + " external_secret: \"shh\"\n"},
{ {
name: "from file", yaml: externalCache + " external_secret_file: \"$DIR/cache.secret\"\n", name: "from file", yaml: externalCache + " external_secret_file: '$DIR/cache.secret'\n",
fileContent: " s3cr3t\n", wantSecret: "s3cr3t", fileContent: " s3cr3t\n", wantSecret: "s3cr3t",
}, },
{ {
name: "from file when disabled", yaml: "\ncache:\n enabled: false\n external_secret_file: \"$DIR/cache.secret\"\n", name: "from file when disabled", yaml: "\ncache:\n enabled: false\n external_secret_file: '$DIR/cache.secret'\n",
fileContent: "s3cr3t", wantSecret: "s3cr3t", fileContent: "s3cr3t", wantSecret: "s3cr3t",
}, },
{ {
name: "inline and file conflict", yaml: externalCache + " external_secret: \"inline\"\n external_secret_file: \"$DIR/cache.secret\"\n", name: "inline and file conflict", yaml: externalCache + " external_secret: \"inline\"\n external_secret_file: '$DIR/cache.secret'\n",
fileContent: "s3cr3t", wantErr: "both set", fileContent: "s3cr3t", wantErr: "both set",
}, },
{name: "missing file", yaml: externalCache + " external_secret_file: \"$DIR/absent.secret\"\n", wantErr: "read cache.external_secret_file"}, {name: "missing file", yaml: externalCache + " external_secret_file: '$DIR/absent.secret'\n", wantErr: "read cache.external_secret_file"},
{ {
name: "empty file", yaml: externalCache + " external_secret_file: \"$DIR/cache.secret\"\n", name: "empty file", yaml: externalCache + " external_secret_file: '$DIR/cache.secret'\n",
fileContent: "\n \n", wantErr: "contains no secret", fileContent: "\n \n", wantErr: "contains no secret",
}, },
} { } {
+8
View File
@@ -0,0 +1,8 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build !windows
package process
func UTF8Console() {}
+11
View File
@@ -0,0 +1,11 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package process
import "golang.org/x/sys/windows"
func UTF8Console() { // per step, as GitHub's runner runs chcp 65001 before its steps
_ = windows.SetConsoleCP(65001)
_ = windows.SetConsoleOutputCP(65001)
}
+1 -3
View File
@@ -11,9 +11,7 @@ import (
"golang.org/x/sys/windows" "golang.org/x/sys/windows"
) )
// Group is a job-scoped Windows Job Object holding every process the job's steps // Group holds tracked step processes in a job-scoped Windows Job Object.
// start; closing it terminates whatever is still assigned, reaching orphans the
// per-step Killer misses (a completed step's leftover has no parent to walk from).
type Group struct { type Group struct {
mu sync.Mutex mu sync.Mutex
job windows.Handle job windows.Handle
+4 -48
View File
@@ -4,69 +4,25 @@
package process package process
import ( import (
"fmt"
"os"
"os/exec" "os/exec"
"path/filepath"
"strconv"
"strings"
"testing" "testing"
"time" "time"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
// startOrphanMaker starts a process that spawns a detached, long-lived child and
// exits, leaving the child parentless. Returns the cmd and the child's PID file.
func startOrphanMaker(t *testing.T) (*exec.Cmd, string) {
t.Helper()
pidFile := filepath.Join(t.TempDir(), "child.pid")
script := fmt.Sprintf(
`$c = Start-Process powershell -PassThru -ArgumentList '-NoProfile','-Command','Start-Sleep -Seconds 600'; `+
`Set-Content -LiteralPath %q -Value $c.Id`, pidFile)
cmd := exec.Command("powershell.exe", "-NoProfile", "-Command", script)
require.NoError(t, cmd.Start())
t.Cleanup(func() { _ = cmd.Process.Kill() })
return cmd, pidFile
}
// awaitChildPID waits until the spawned child has reported its PID and is running.
func awaitChildPID(t *testing.T, pidFile string) int {
t.Helper()
var childPID int
require.Eventually(t, func() bool {
b, err := os.ReadFile(pidFile)
if err != nil {
return false
}
s := strings.TrimSpace(string(b))
if s == "" {
return false
}
childPID, _ = strconv.Atoi(s)
return childPID > 0 && processAlive(childPID)
}, 20*time.Second, 200*time.Millisecond, "child process should start")
return childPID
}
// TestGroupClosePropagatesToOrphan covers what the per-step Killer cannot: a
// completed step's orphan, reachable only by closing the job object.
func TestGroupClosePropagatesToOrphan(t *testing.T) { func TestGroupClosePropagatesToOrphan(t *testing.T) {
group, err := NewGroup() group, err := NewGroup()
require.NoError(t, err) require.NoError(t, err)
t.Cleanup(func() { _ = group.Close() }) t.Cleanup(func() { _ = group.Close() })
cmd, pidFile := startOrphanMaker(t) cmd, ready := newTestProcess(t, "orphan", false)
require.NoError(t, cmd.Start())
require.NoError(t, group.Assign(cmd.Process)) require.NoError(t, group.Assign(cmd.Process))
childPID := awaitChildPID(t, pidFile) childPID := ready()
require.NoError(t, cmd.Wait()) // the step process exits cleanly, like a passing step require.NoError(t, cmd.Wait())
require.Eventually(t, func() bool { require.Eventually(t, func() bool {
return !processAlive(cmd.Process.Pid) return !processAlive(cmd.Process.Pid)
}, 20*time.Second, 200*time.Millisecond, "the step process should have exited on its own") }, 20*time.Second, 200*time.Millisecond, "the step process should have exited on its own")
+7 -12
View File
@@ -4,9 +4,6 @@
package process package process
import ( import (
"fmt"
"os/exec"
"path/filepath"
"testing" "testing"
"time" "time"
@@ -29,20 +26,18 @@ func processAlive(pid int) bool {
} }
func TestKillerKillsTree(t *testing.T) { func TestKillerKillsTree(t *testing.T) {
pidFile := filepath.Join(t.TempDir(), "child.pid") cmd, ready := newTestProcess(t, "parent", false)
script := fmt.Sprintf(
`$c = Start-Process powershell -PassThru -ArgumentList '-NoProfile','-Command','Start-Sleep -Seconds 600'; `+
`Set-Content -LiteralPath %q -Value $c.Id; Start-Sleep -Seconds 600`, pidFile)
cmd := exec.Command("powershell.exe", "-NoProfile", "-Command", script)
require.NoError(t, cmd.Start()) require.NoError(t, cmd.Start())
t.Cleanup(func() { _ = cmd.Process.Kill() })
killer, err := NewKiller(cmd.Process) killer, err := NewKiller(cmd.Process)
require.NoError(t, err) require.NoError(t, err)
defer killer.Close() t.Cleanup(func() {
_ = killer.Kill()
_ = killer.Close()
})
childPID := awaitChildPID(t, pidFile) childPID := ready()
require.True(t, processAlive(childPID), "child process should start")
require.NoError(t, killer.Kill()) require.NoError(t, killer.Kill())
+1 -3
View File
@@ -7,8 +7,6 @@ package process
import "context" import "context"
// KillProcessesWithCWDUnder is a no-op outside Windows: an open handle does not func KillProcessesWithPaths(_ctx context.Context, _dirs []string, _trackingID string) (int, error) {
// block os.RemoveAll there.
func KillProcessesWithCWDUnder(_ context.Context, _ []string) (int, error) {
return 0, nil return 0, nil
} }
+147 -59
View File
@@ -8,16 +8,14 @@ import (
"errors" "errors"
"fmt" "fmt"
"os" "os"
"strings"
"unsafe" "unsafe"
"golang.org/x/sys/windows" "golang.org/x/sys/windows"
) )
// KillProcessesWithCWDUnder terminates every process in the runner's session func KillProcessesWithPaths(ctx context.Context, dirs []string, trackingID string) (int, error) {
// whose cwd is one of dirs or below, catching leftovers the path scan misses if len(dirs) == 0 || trackingID == "" {
// (Win32_Process exposes no cwd) that still pin a handle. Best-effort.
func KillProcessesWithCWDUnder(ctx context.Context, dirs []string) (int, error) {
if len(dirs) == 0 {
return 0, nil return 0, nil
} }
@@ -38,7 +36,6 @@ func KillProcessesWithCWDUnder(ctx context.Context, dirs []string) (int, error)
if ctx.Err() != nil { if ctx.Err() != nil {
break break
} }
// Never touch ourselves, System Idle (0) or System (4).
if pid == self || pid == 0 || pid == 4 { if pid == self || pid == 0 || pid == 4 {
continue continue
} }
@@ -46,7 +43,7 @@ func KillProcessesWithCWDUnder(ctx context.Context, dirs []string) (int, error)
if err := windows.ProcessIdToSessionId(pid, &session); err != nil || session != ownSession { if err := windows.ProcessIdToSessionId(pid, &session); err != nil || session != ownSession {
continue continue
} }
ok, err := terminateIfCWDUnder(pid, dirs) ok, err := terminateIfPathsMatch(pid, dirs, trackingID)
if err != nil { if err != nil {
errs = append(errs, err) errs = append(errs, err)
continue continue
@@ -59,29 +56,50 @@ func KillProcessesWithCWDUnder(ctx context.Context, dirs []string) (int, error)
return killed, errors.Join(errs...) return killed, errors.Join(errs...)
} }
// terminateIfCWDUnder opens pid once and both reads its cwd and terminates it func terminateIfPathsMatch(pid uint32, dirs []string, trackingID string) (bool, error) {
// through that one handle, so a PID recycled between the two cannot be hit. handle, err := windows.OpenProcess(windows.PROCESS_QUERY_INFORMATION|windows.PROCESS_VM_READ|windows.PROCESS_TERMINATE, false, pid)
func terminateIfCWDUnder(pid uint32, dirs []string) (bool, error) {
h, err := windows.OpenProcess(windows.PROCESS_QUERY_INFORMATION|windows.PROCESS_VM_READ|windows.PROCESS_TERMINATE, false, pid)
if err != nil { if err != nil {
return false, nil // best-effort: gone, higher integrity, or another user
}
defer func() { _ = windows.CloseHandle(h) }()
cwd, err := processCWDHandle(h)
if err != nil || cwd == "" {
return false, nil return false, nil
} }
defer func() { _ = windows.CloseHandle(handle) }()
for _, d := range dirs { params, err := processParametersHandle(handle)
if dirContainsPath(d, cwd) { if err != nil {
if err := windows.TerminateProcess(h, 1); err != nil { return false, nil
return false, fmt.Errorf("terminate pid %d: %w", pid, err) }
matches := false
fields:
for _, field := range []struct {
offset uintptr
wow64 uintptr
command bool
}{
{unsafe.Offsetof(windows.RTL_USER_PROCESS_PARAMETERS{}.CurrentDirectory), 0x24, false},
{unsafe.Offsetof(windows.RTL_USER_PROCESS_PARAMETERS{}.ImagePathName), 0x38, false},
{unsafe.Offsetof(windows.RTL_USER_PROCESS_PARAMETERS{}.CommandLine), 0x40, true},
} {
value, err := params.unicodeString(handle, field.offset, field.wow64)
if err != nil {
continue
}
for _, dir := range dirs {
if !field.command && dirContainsPath(dir, value) || field.command && strings.Contains(strings.ToLower(value), strings.ToLower(strings.TrimRight(dir, `\/`)+`\`)) {
matches = true
break fields
} }
return true, nil
} }
} }
return false, nil if !matches {
return false, nil
}
value, err := params.environmentVariable(handle, "RUNNER_TRACKING_ID")
if err != nil || value != trackingID {
return false, nil
}
if err := windows.TerminateProcess(handle, 1); err != nil {
return false, fmt.Errorf("terminate pid %d: %w", pid, err)
}
return true, nil
} }
// listProcessIDs returns the PIDs of every process in a Toolhelp snapshot. // listProcessIDs returns the PIDs of every process in a Toolhelp snapshot.
@@ -107,65 +125,135 @@ func listProcessIDs() ([]uint32, error) {
return pids, nil return pids, nil
} }
// processCWDHandle reads a process's working directory from its PEB (walking type remoteParameters struct {
// PEB -> ProcessParameters -> CurrentDirectory.DosPath). h must carry address uintptr
// PROCESS_QUERY_INFORMATION|PROCESS_VM_READ. wow64 bool
func processCWDHandle(h windows.Handle) (string, error) { }
func processParametersHandle(handle windows.Handle) (remoteParameters, error) {
var pbi windows.PROCESS_BASIC_INFORMATION var pbi windows.PROCESS_BASIC_INFORMATION
var retLen uint32 var retLen uint32
if err := windows.NtQueryInformationProcess(h, windows.ProcessBasicInformation, unsafe.Pointer(&pbi), uint32(unsafe.Sizeof(pbi)), &retLen); err != nil { if err := windows.NtQueryInformationProcess(handle, windows.ProcessBasicInformation, unsafe.Pointer(&pbi), uint32(unsafe.Sizeof(pbi)), &retLen); err != nil {
return "", err return remoteParameters{}, err
}
if pbi.PebBaseAddress == nil {
return "", errors.New("nil PEB base address")
} }
pebAddr := uintptr(unsafe.Pointer(pbi.PebBaseAddress)) pebAddr := uintptr(unsafe.Pointer(pbi.PebBaseAddress))
wow64 := false
paramsAddr, err := readRemotePtr(h, pebAddr+unsafe.Offsetof(windows.PEB{}.ProcessParameters)) if unsafe.Sizeof(uintptr(0)) == 8 {
if err != nil || paramsAddr == 0 { var peb32 uintptr
return "", err if err := windows.NtQueryInformationProcess(handle, windows.ProcessWow64Information, unsafe.Pointer(&peb32), uint32(unsafe.Sizeof(peb32)), &retLen); err != nil {
return remoteParameters{}, err
}
if peb32 != 0 {
pebAddr = peb32
wow64 = true
}
} }
if pebAddr == 0 {
return remoteParameters{}, errors.New("nil PEB base address")
}
params := remoteParameters{wow64: wow64}
address, err := params.pointer(handle, pebAddr+params.offset(unsafe.Offsetof(windows.PEB{}.ProcessParameters), 0x10))
if err != nil {
return remoteParameters{}, err
}
if address == 0 {
return remoteParameters{}, errors.New("nil process parameters")
}
params.address = address
return params, nil
}
// DosPath is the leading NTUnicodeString of CURDIR, so it shares CurrentDirectory's address. func (p remoteParameters) offset(native, wow64 uintptr) uintptr {
dosPathAddr := paramsAddr + unsafe.Offsetof(windows.RTL_USER_PROCESS_PARAMETERS{}.CurrentDirectory) if p.wow64 {
length, err := readRemoteU16(h, dosPathAddr+unsafe.Offsetof(windows.NTUnicodeString{}.Length)) return wow64
}
return native
}
func (p remoteParameters) pointer(handle windows.Handle, address uintptr) (uintptr, error) {
var value uintptr
size := unsafe.Sizeof(value)
if p.wow64 {
size = 4
}
var read uintptr
if err := windows.ReadProcessMemory(handle, address, (*byte)(unsafe.Pointer(&value)), size, &read); err != nil {
return 0, err
}
if read != size {
return 0, errors.New("short pointer read")
}
return value, nil
}
func (p remoteParameters) unicodeString(handle windows.Handle, native, wow64 uintptr) (string, error) {
address := p.address + p.offset(native, wow64)
length, err := readRemoteU16(handle, address)
if err != nil || length == 0 { if err != nil || length == 0 {
return "", err return "", err
} }
bufAddr, err := readRemotePtr(h, dosPathAddr+unsafe.Offsetof(windows.NTUnicodeString{}.Buffer)) buffer, err := p.pointer(handle, address+p.offset(unsafe.Offsetof(windows.NTUnicodeString{}.Buffer), 4))
if err != nil || bufAddr == 0 { if err != nil {
return "", err return "", err
} }
value := make([]uint16, (int(length)+1)/2)
u16 := make([]uint16, (length+1)/2) // round up so an odd Length cannot overflow the buffer var read uintptr
var n uintptr if err := windows.ReadProcessMemory(handle, buffer, (*byte)(unsafe.Pointer(&value[0])), uintptr(length), &read); err != nil {
if err := windows.ReadProcessMemory(h, bufAddr, (*byte)(unsafe.Pointer(&u16[0])), uintptr(length), &n); err != nil {
return "", err return "", err
} }
if n != uintptr(length) { if read != uintptr(length) {
return "", errors.New("short read of working directory") return "", errors.New("short string read")
} }
return windows.UTF16ToString(u16), nil return windows.UTF16ToString(value), nil
} }
// readRemotePtr reads a single pointer-sized value from another process. func (p remoteParameters) environmentVariable(handle windows.Handle, name string) (string, error) {
func readRemotePtr(h windows.Handle, addr uintptr) (uintptr, error) { address, err := p.pointer(handle, p.address+p.offset(unsafe.Offsetof(windows.RTL_USER_PROCESS_PARAMETERS{}.Environment), 0x48))
var v uintptr if err != nil || address == 0 {
var n uintptr return "", err
if err := windows.ReadProcessMemory(h, addr, (*byte)(unsafe.Pointer(&v)), unsafe.Sizeof(v), &n); err != nil {
return 0, err
} }
if n != unsafe.Sizeof(v) { var entry []uint16
return 0, errors.New("short pointer read") for consumed := uintptr(0); consumed < 32*1024*1024; {
var region windows.MemoryBasicInformation
if err := windows.VirtualQueryEx(handle, address+consumed, &region, unsafe.Sizeof(region)); err != nil {
return "", err
}
size := min(uintptr(4096), region.BaseAddress+region.RegionSize-address-consumed)
if size < 2 {
return "", errors.New("invalid environment region")
}
buffer := make([]uint16, size/2)
var read uintptr
if err := windows.ReadProcessMemory(handle, address+consumed, (*byte)(unsafe.Pointer(&buffer[0])), uintptr(len(buffer)*2), &read); err != nil {
return "", err
}
if read != uintptr(len(buffer)*2) {
return "", errors.New("short environment read")
}
for _, value := range buffer {
if value != 0 {
entry = append(entry, value)
continue
}
if len(entry) == 0 {
return "", nil
}
key, value, ok := strings.Cut(windows.UTF16ToString(entry), "=")
if ok && strings.EqualFold(key, name) {
return value, nil
}
entry = entry[:0]
}
consumed += read
} }
return v, nil return "", errors.New("environment exceeds read limit")
} }
// readRemoteU16 reads a single uint16 from another process. // readRemoteU16 reads a single uint16 from another process.
func readRemoteU16(h windows.Handle, addr uintptr) (uint16, error) { func readRemoteU16(handle windows.Handle, addr uintptr) (uint16, error) {
var v uint16 var v uint16
var n uintptr var n uintptr
if err := windows.ReadProcessMemory(h, addr, (*byte)(unsafe.Pointer(&v)), unsafe.Sizeof(v), &n); err != nil { if err := windows.ReadProcessMemory(handle, addr, (*byte)(unsafe.Pointer(&v)), unsafe.Sizeof(v), &n); err != nil {
return 0, err return 0, err
} }
if n != unsafe.Sizeof(v) { if n != unsafe.Sizeof(v) {
+53 -37
View File
@@ -7,21 +7,21 @@ import (
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"runtime"
"syscall" "syscall"
"testing" "testing"
"time" "time"
"unsafe"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"golang.org/x/sys/windows" "golang.org/x/sys/windows"
) )
// startSleeperIn launches a long-running child whose cwd is dir but whose func startSleeperIn(t *testing.T, system, dir, trackingID string) *exec.Cmd {
// executable lives outside it — what the path scan cannot see.
func startSleeperIn(t *testing.T, dir string) *exec.Cmd {
t.Helper() t.Helper()
ping := filepath.Join(os.Getenv("SystemRoot"), "System32", "ping.exe") cmd := exec.Command(filepath.Join(os.Getenv("SystemRoot"), system, "ping.exe"), "-n", "60", "127.0.0.1")
cmd := exec.Command(ping, "-n", "60", "127.0.0.1")
cmd.Dir = dir cmd.Dir = dir
cmd.Env = append(os.Environ(), "RUNNER_TRACKING_ID="+trackingID)
cmd.SysProcAttr = &syscall.SysProcAttr{HideWindow: true} cmd.SysProcAttr = &syscall.SysProcAttr{HideWindow: true}
require.NoError(t, cmd.Start()) require.NoError(t, cmd.Start())
t.Cleanup(func() { t.Cleanup(func() {
@@ -31,19 +31,19 @@ func startSleeperIn(t *testing.T, dir string) *exec.Cmd {
return cmd return cmd
} }
// processCWD opens pid and reads its working directory. Test-only; the reaper
// shares one handle across read and kill via processCWDHandle.
func processCWD(pid uint32) (string, error) { func processCWD(pid uint32) (string, error) {
h, err := windows.OpenProcess(windows.PROCESS_QUERY_INFORMATION|windows.PROCESS_VM_READ, false, pid) h, err := windows.OpenProcess(windows.PROCESS_QUERY_INFORMATION|windows.PROCESS_VM_READ, false, pid)
if err != nil { if err != nil {
return "", err return "", err
} }
defer func() { _ = windows.CloseHandle(h) }() defer func() { _ = windows.CloseHandle(h) }()
return processCWDHandle(h) params, err := processParametersHandle(h)
if err != nil {
return "", err
}
return params.unicodeString(h, unsafe.Offsetof(windows.RTL_USER_PROCESS_PARAMETERS{}.CurrentDirectory), 0x24)
} }
// awaitCWD waits until pid's PEB reports a readable cwd (not populated the
// instant the process starts) and returns it.
func awaitCWD(t *testing.T, pid int) string { func awaitCWD(t *testing.T, pid int) string {
t.Helper() t.Helper()
var cwd string var cwd string
@@ -55,36 +55,52 @@ func awaitCWD(t *testing.T, pid int) string {
return cwd return cwd
} }
func TestProcessCWDReadsWorkingDirectory(t *testing.T) { func TestKillProcessesWithPathsTracking(t *testing.T) {
dir := t.TempDir() for _, system := range []string{"System32", "SysWOW64"} {
cmd := startSleeperIn(t, dir) for _, trackingID := range []string{"leftover-test", "", "another-job"} {
cwd := awaitCWD(t, cmd.Process.Pid) t.Run(system+"/tracking="+trackingID, func(t *testing.T) {
require.True(t, dirContainsPath(dir, cwd), "cwd %q should be within %q", cwd, dir) if system == "SysWOW64" {
if runtime.GOARCH != "amd64" && runtime.GOARCH != "arm64" {
t.Skip("needs a 64-bit runner")
}
if _, err := os.Stat(filepath.Join(os.Getenv("SystemRoot"), system, "ping.exe")); err != nil {
t.Skip("32-bit Windows tools unavailable")
}
}
dir := t.TempDir()
cmd := startSleeperIn(t, system, dir, trackingID)
require.True(t, dirContainsPath(dir, awaitCWD(t, cmd.Process.Pid)))
killed, err := KillProcessesWithPaths(t.Context(), []string{t.TempDir()}, "leftover-test")
require.NoError(t, err)
require.Zero(t, killed)
killed, err = KillProcessesWithPaths(t.Context(), []string{dir}, "leftover-test")
require.NoError(t, err)
if trackingID == "leftover-test" {
require.Equal(t, 1, killed)
require.Eventually(t, func() bool {
return !processAlive(cmd.Process.Pid)
}, 5*time.Second, 50*time.Millisecond)
} else {
require.Zero(t, killed)
require.True(t, processAlive(cmd.Process.Pid))
}
})
}
}
} }
func TestKillProcessesWithCWDUnderTerminatesMatch(t *testing.T) { func TestKillProcessesWithPathsCommandLine(t *testing.T) {
dir := t.TempDir() workspace := t.TempDir()
// Executable outside dir and arguments carry no workspace path, so only the cmd, ready := newTestProcess(t, "child", false)
// working directory links this process to dir. cmd.Dir = t.TempDir()
cmd := startSleeperIn(t, dir) cmd.Args = append(cmd.Args, filepath.Join(workspace, "reference"))
awaitCWD(t, cmd.Process.Pid) cmd.Env = append(cmd.Env, "RUNNER_TRACKING_ID=command-test")
require.NoError(t, cmd.Start())
killed, err := KillProcessesWithCWDUnder(t.Context(), []string{dir}) ready()
killed, err := KillProcessesWithPaths(t.Context(), []string{workspace}, "command-test")
require.NoError(t, err) require.NoError(t, err)
require.GreaterOrEqual(t, killed, 1) require.Equal(t, 1, killed)
require.Eventually(t, func() bool { require.Eventually(t, func() bool {
return !processAlive(cmd.Process.Pid) return !processAlive(cmd.Process.Pid)
}, 5*time.Second, 50*time.Millisecond, "matched process should exit") }, 5*time.Second, 50*time.Millisecond)
}
func TestKillProcessesWithCWDUnderSparesUnrelated(t *testing.T) {
workspace := t.TempDir()
elsewhere := t.TempDir()
cmd := startSleeperIn(t, elsewhere)
awaitCWD(t, cmd.Process.Pid)
_, err := KillProcessesWithCWDUnder(t.Context(), []string{workspace})
require.NoError(t, err)
require.True(t, processAlive(cmd.Process.Pid), "process outside owned dirs must be spared")
} }
+4 -3
View File
@@ -32,7 +32,8 @@ const treeKillWaitDelay = 10 * time.Second
// Callers still set cmd.SysProcAttr (via SysProcAttr) themselves, because the // Callers still set cmd.SysProcAttr (via SysProcAttr) themselves, because the
// value differs between the plain and PTY execution paths. // value differs between the plain and PTY execution paths.
type TreeKill struct { type TreeKill struct {
killer atomic.Pointer[Killer] killer atomic.Pointer[Killer]
KeepLeftovers bool
} }
// NewTreeKill sets cmd.Cancel and cmd.WaitDelay. Call it before cmd.Start, then // NewTreeKill sets cmd.Cancel and cmd.WaitDelay. Call it before cmd.Start, then
@@ -72,13 +73,13 @@ func (t *TreeKill) Capture(p *os.Process) (*Killer, error) {
return k, nil return k, nil
} }
// Wait is cmd.Wait, but like GitHub's runner a leftover holding a successful command's output does not fail it, and is killed only on Windows. // Wait is cmd.Wait, but like GitHub's runner a leftover holding a successful command's output does not fail it.
func (t *TreeKill) Wait(ctx context.Context, cmd *exec.Cmd) error { func (t *TreeKill) Wait(ctx context.Context, cmd *exec.Cmd) error {
err := cmd.Wait() err := cmd.Wait()
if !errors.Is(err, exec.ErrWaitDelay) || ctx.Err() != nil { if !errors.Is(err, exec.ErrWaitDelay) || ctx.Err() != nil {
return err return err
} }
if runtime.GOOS == "windows" { if runtime.GOOS == "windows" && !t.KeepLeftovers {
_ = t.killer.Load().Kill() _ = t.killer.Load().Kill()
} }
return nil return nil
+86 -3
View File
@@ -5,8 +5,11 @@ package process
import ( import (
"context" "context"
"fmt"
"io" "io"
"os"
"os/exec" "os/exec"
"os/signal"
"runtime" "runtime"
"testing" "testing"
"time" "time"
@@ -60,11 +63,10 @@ func TestTreeKillWaitWithLeftoverHoldingOutput(t *testing.T) {
} }
func TestTreeKillCaptureStoresKiller(t *testing.T) { func TestTreeKillCaptureStoresKiller(t *testing.T) {
cmd := exec.CommandContext(t.Context(), "sleep", "10") cmd, ready := newTestProcess(t, "child", false)
cmd.SysProcAttr = SysProcAttr("", false)
tk := NewTreeKill(cmd) tk := NewTreeKill(cmd)
require.NoError(t, cmd.Start()) require.NoError(t, cmd.Start())
defer func() { _ = cmd.Wait() }() ready()
killer, err := tk.Capture(cmd.Process) killer, err := tk.Capture(cmd.Process)
require.NoError(t, err) require.NoError(t, err)
@@ -72,3 +74,84 @@ func TestTreeKillCaptureStoresKiller(t *testing.T) {
require.NoError(t, cmd.Cancel()) require.NoError(t, cmd.Cancel())
require.NoError(t, killer.Close()) require.NoError(t, killer.Close())
} }
func newTestProcess(t *testing.T, mode string, copiedOutput bool) (*exec.Cmd, func() int) {
t.Helper()
ctx, cancel := context.WithTimeout(t.Context(), 20*time.Second)
t.Cleanup(cancel)
cmd := exec.CommandContext(ctx, os.Args[0], "-test.run=^TestProcessHelper$")
cmd.Env = append(os.Environ(), "GITEA_PROCESS_HELPER="+mode)
cmd.SysProcAttr = SysProcAttr("", false)
cmd.Stderr = os.Stderr
input, err := cmd.StdinPipe()
require.NoError(t, err)
var output io.ReadCloser
if copiedOutput {
var writer *io.PipeWriter
output, writer = io.Pipe()
cmd.Stdout = writer
} else {
output, err = cmd.StdoutPipe()
require.NoError(t, err)
}
stop := context.AfterFunc(ctx, func() {
_ = input.Close()
_ = output.Close()
})
t.Cleanup(func() {
stop()
_ = input.Close()
_ = output.Close()
if cmd.Process != nil {
_ = cmd.Process.Kill()
_ = cmd.Wait()
}
})
return cmd, func() int {
t.Helper()
_, err := input.Write([]byte{1})
require.NoError(t, err)
var childPID int
_, err = fmt.Fscan(output, &childPID)
require.NoError(t, err, "process should report its PID")
require.Positive(t, childPID)
return childPID
}
}
func TestProcessHelper(t *testing.T) {
mode := os.Getenv("GITEA_PROCESS_HELPER")
switch mode {
case "":
return
case "orphan-child":
interrupt := make(chan os.Signal, 1)
signal.Notify(interrupt, os.Interrupt)
_, err := fmt.Fprintln(os.Stdout, os.Getpid())
require.NoError(t, err)
<-interrupt
case "child":
_, err := fmt.Fprintln(os.Stdout, os.Getpid())
require.NoError(t, err)
_, err = io.Copy(io.Discard, os.Stdin)
require.NoError(t, err)
default:
var start [1]byte
_, err := io.ReadFull(os.Stdin, start[:])
require.NoError(t, err)
child := exec.Command(os.Args[0], "-test.run=^TestProcessHelper$")
child.Env = append(os.Environ(), "GITEA_PROCESS_HELPER=child")
if mode == "orphan" {
child.Env = append(os.Environ(), "GITEA_PROCESS_HELPER=orphan-child")
}
child.Stdin = os.Stdin
child.Stdout = os.Stdout
child.Stderr = os.Stderr
require.NoError(t, child.Start())
if mode == "orphan" {
os.Exit(0)
}
require.NoError(t, child.Wait())
}
os.Exit(0)
}
@@ -0,0 +1,45 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package process
import (
"testing"
"time"
"github.com/stretchr/testify/require"
"golang.org/x/sys/windows"
)
func TestTreeKillWaitTrackingLeftoverOutput(t *testing.T) {
for _, keep := range []bool{false, true} {
t.Run(map[bool]string{false: "tracked", true: "opted out"}[keep], func(t *testing.T) {
cmd, ready := newTestProcess(t, "orphan", true)
tree := NewTreeKill(cmd)
tree.KeepLeftovers = keep
cmd.WaitDelay = 50 * time.Millisecond
require.NoError(t, cmd.Start())
killer, err := tree.Capture(cmd.Process)
require.NoError(t, err)
defer killer.Close()
childPID := ready()
handle, err := windows.OpenProcess(windows.SYNCHRONIZE|windows.PROCESS_TERMINATE, false, uint32(childPID))
require.NoError(t, err)
t.Cleanup(func() {
_ = windows.TerminateProcess(handle, 1)
_, _ = windows.WaitForSingleObject(handle, 5000)
_ = windows.CloseHandle(handle)
})
require.NoError(t, tree.Wait(t.Context(), cmd))
if keep {
result, err := windows.WaitForSingleObject(handle, 0)
require.NoError(t, err)
require.EqualValues(t, windows.WAIT_TIMEOUT, result)
} else {
result, err := windows.WaitForSingleObject(handle, 5000)
require.NoError(t, err)
require.EqualValues(t, windows.WAIT_OBJECT_0, result)
}
})
}
}
+29
View File
@@ -0,0 +1,29 @@
$ErrorActionPreference = 'Stop'
$ProgressPreference = 'SilentlyContinue'
$shared = '\\host.lan\Data'
try {
Write-Output '==> Setting up Windows'
Set-ItemProperty HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem LongPathsEnabled 1
foreach ($tool in @(
@('git.7z.exe', 'https://github.com/git-for-windows/git/releases/download/v2.56.0.windows.3/PortableGit-2.56.0.3-64-bit.7z.exe', '4fc27b779829a36df77911e3f4dea09c66c790dd6c5f7dfa3b6a5479f1fe354f'),
@('node.zip', 'https://nodejs.org/dist/v24.22.1/node-v24.22.1-win-x64.zip', 'd2046d28c2c3dcd211ac5145c0bf30e2047ef9dfb306693c8c6e2eccd8b95fa5'),
@('pwsh.zip', 'https://github.com/PowerShell/PowerShell/releases/download/v7.6.8/PowerShell-7.6.8-win-x64.zip', 'b9f074f923c499b51ec99d313da8e9ae90d17a9f3adf00c73d6189f34d908528'))) {
Write-Output "==> Downloading $($tool[1])"
Invoke-WebRequest -UseBasicParsing $tool[1] -OutFile "$env:TEMP\$($tool[0])"
if ((Get-FileHash "$env:TEMP\$($tool[0])").Hash -ne $tool[2]) { throw "checksum mismatch for $($tool[1])" }
}
Write-Output '==> Extracting git, node and pwsh'
New-Item -ItemType Directory C:\pwsh | Out-Null
@(
Start-Process "$env:TEMP\git.7z.exe" -ArgumentList '-oC:\git', '-y' -PassThru
Start-Process tar.exe -ArgumentList '-xf', "$env:TEMP\node.zip", '-C', 'C:\' -PassThru -NoNewWindow
Start-Process tar.exe -ArgumentList '-xf', "$env:TEMP\pwsh.zip", '-C', 'C:\pwsh' -PassThru -NoNewWindow
) | Wait-Process
Rename-Item (Resolve-Path C:\node-*-win-x64).Path C:\node
$env:Path = "C:\git\cmd;C:\git\bin;C:\pwsh;C:\node;$env:Path"
Get-Command git, bash, node, pwsh | Out-Null
} catch {
$_ | Out-String | Tee-Object "$shared\error.txt"
Set-Content "$shared\exitcode" 1
exit 1
}
+31
View File
@@ -0,0 +1,31 @@
$ErrorActionPreference = 'Stop'
$ProgressPreference = 'SilentlyContinue'
$results = '\\host.lan\Data'
$root = $PSScriptRoot
try {
New-Item -ItemType Directory "$root\logs" | Out-Null
$env:Path = "C:\git\cmd;C:\git\bin;C:\pwsh;C:\node;$env:Path"
git config --global --add safe.directory C:/OEM/src # setup copied it as SYSTEM, the tests run as the logged-on user
$failed = 0
Write-Output "==> Running $((Get-Content "$root\tests.txt").Count) test binaries on $([Environment]::ProcessorCount) cores"
Get-Content "$root\tests.txt" | ForEach-Object -ThrottleLimit ([Environment]::ProcessorCount) -Parallel {
$binary, $package = $_ -split ' ', 2
$started = Get-Date
Set-Location "$using:root\src\$package"
cmd.exe /d /c "$using:root\bin\$binary -test.v -test.timeout=60m -test.parallel=8 > $using:root\logs\$($package -replace '/', '-').log 2>&1"
[pscustomobject]@{ Package = $package; ExitCode = $LASTEXITCODE; Seconds = [int]((Get-Date) - $started).TotalSeconds }
} | ForEach-Object {
$line = "ok $($_.Package) $($_.Seconds)s"
if ($_.ExitCode -ne 0) {
$line = "FAIL $($_.Package) $($_.Seconds)s exit $($_.ExitCode)"
$failed = 1
}
Add-Content "$results\summary.txt" $line
}
Copy-Item "$root\logs\*" $results
} catch {
$_ | Out-String | Tee-Object "$results\error.txt"
$failed = 1
}
Set-Content "$results\exitcode" $failed
+90
View File
@@ -0,0 +1,90 @@
#!/usr/bin/env bash
set -euo pipefail
image="${WINDOWS_TEST_IMAGE:?}"
timeout="${WINDOWS_TEST_TIMEOUT:-165}"
name="gitea-runner-windows-test-$$"
work="$(mktemp -d)"
cleanup() {
docker rm -fv "$name" >/dev/null 2>&1 || true
rm -rf "$work"
}
trap cleanup EXIT
if [ "$(docker info -f '{{.Architecture}}')" != x86_64 ]; then
echo "error: the Windows test VM needs an amd64 docker host" >&2
exit 1
fi
echo "==> Building Windows test binaries"
oem="$work/oem"
mkdir -p "$oem/bin" "$oem/src" "$work/shared"
module="$(go list -m)"
packages="$(GOOS=windows go list -f '{{if or .TestGoFiles .XTestGoFiles}}{{.ImportPath}}{{end}}' ./...)"
GOOS=windows GOARCH=amd64 go test -c -ldflags='-s -w' -o "$oem/bin/" $packages
for package in $packages; do
echo "${package##*/}.test.exe ${package#"$module"/}"
done > "$oem/tests.txt"
git ls-files -z --cached --others --exclude-standard | while IFS= read -r -d '' file; do
if [ -e "$file" ]; then printf '%s\0' "$file"; fi
done | tar --null -T - -cf - | tar -xf - -C "$oem/src"
git -C "$oem/src" init -q -b main # a repository like the CI checkout, for the tests that read the github context from it
git -C "$oem/src" add -Af .
git -C "$oem/src" -c user.name=test -c user.email=test@localhost -c commit.gpgsign=false -c core.hooksPath=/dev/null commit -qm test
git -C "$oem/src" remote add origin https://gitea.com/gitea/runner
cp scripts/test-windows-setup.ps1 scripts/test-windows.ps1 "$oem/"
sed 's/$/\r/' > "$oem/install.bat" <<'EOF' # a policy turns off real-time scanning, AMSI blocks scripts that do it
@echo off
set log=\\host.lan\Data\install.log
echo ==^> install.bat started at %TIME% >> %log%
reg.exe add "HKLM\SOFTWARE\Policies\Microsoft\Windows Defender\Real-Time Protection" /v DisableRealtimeMonitoring /t REG_DWORD /d 1 /f > nul
powershell.exe -NoProfile -ExecutionPolicy Bypass -File C:\OEM\test-windows-setup.ps1 >> %log% 2>&1
echo ==^> setup exited with %ERRORLEVEL% >> %log%
if %ERRORLEVEL% equ 0 C:\pwsh\pwsh.exe -NoProfile -ExecutionPolicy Bypass -File C:\OEM\test-windows.ps1 >> %log% 2>&1
if not exist \\host.lan\Data\exitcode (echo 1)> \\host.lan\Data\exitcode
shutdown.exe /s /t 600
EOF
kvm=Y cores="$(nproc)"
if ! docker run --rm --privileged --entrypoint sh "$image" -c 'test -w /dev/kvm'; then
kvm=N cores=4 # more emulated vCPUs slow Windows down
echo "==> The docker host has no usable /dev/kvm, Windows runs emulated and much slower"
fi
echo "==> Installing Windows"
docker create --rm --name "$name" --privileged -e VERSION=2025 -e EDITION=core -e KVM="$kvm" -e CPU_CORES="$cores" -e RAM_SIZE=8G \
-e DISK_SIZE=32G -e DISK_CACHE=unsafe -e DISK_IO=threads --entrypoint timeout "$image" "$((timeout + 10))m" /usr/bin/tini -s /run/entry.sh >/dev/null # the timeout stops the VM even when this script is killed before its cleanup runs
docker cp "$oem" "$name:/oem"
docker cp "$work/shared" "$name:/shared"
docker start "$name" >/dev/null
docker logs -f "$name" &
while sleep 10 && docker container inspect "$name" >/dev/null 2>&1; do
if ((SECONDS % 300 < 10)); then
echo "==> $((SECONDS / 60))m elapsed, VM disk $(docker exec "$name" du -sh /storage | cut -f1)"
fi
done &
docker exec "$name" sh -c 'cd /shared && touch install.log summary.txt && tail -qf install.log summary.txt' &
if ! timeout "${timeout}m" docker exec "$name" sh -c 'until [ -f /shared/exitcode ]; do sleep 10; done'; then
echo "Windows stopped or did not finish the tests within $timeout minutes" >&2
exit 1
fi
results="$work/results"
docker cp "$name:/shared" "$results"
tr -d '\r' < "$results/summary.txt" | while read -r result package _; do
if [ "$result" = FAIL ]; then
echo "::group::$package"
cat "$results/${package//\//-}.log"
echo "::endgroup::"
fi
done
if [ -f "$results/error.txt" ]; then
cat "$results/error.txt" >&2
fi
echo "==> Slowest tests"
find "$results" -name '*.log' -exec cat {} + | tr -d '\r' | sed -nE 's/^--- (PASS|FAIL): (\S+) \(([0-9.]+)s\)$/\3 \2/p' | sort -rn | awk 'NR <= 20'
echo "==> Packages"
tr -d '\r' < "$results/summary.txt"
exit "$(tr -d '\r\n' < "$results/exitcode")"