1
0
mirror of https://gitea.com/gitea/act synced 2026-05-01 01:27:48 +02:00

feat: Add support for Dynamic Matrix Strategies with Job Outputs (#149)

# Matrix Strategy: Support for Scalar Values and Template Expressions

## 🎯 Overview

This Pull Request implements support for **scalar values and template expressions** in matrix strategies. Previously, every matrix value had to be declared as a YAML array — a bare scalar like `os: ubuntu-latest` or `version: ${{ fromJSON(...) }}` caused silent failures. Now, scalars are automatically wrapped into single-element arrays, enabling the `${{ fromJSON(...) }}` pattern that is commonly used in GitHub Actions workflows for dynamic matrix generation.

## ⚠️ Semantic Changes

Two subtle behavior changes are introduced that are worth understanding:

### 1. Scalar matrix values now produce a 1-element matrix expansion

**Before this PR:** A scalar value like

```yaml
strategy:
  matrix:
    os: ubuntu-latest   # no brackets
```

failed to decode as `map[string][]interface{}`, so `Matrix()` returned `nil`. The job ran **once, without any matrix context** (`matrix.os` was undefined).

**After this PR:** The same input is wrapped to `["ubuntu-latest"]`. The job runs **one matrix iteration** with `matrix.os = "ubuntu-latest"` set.

> **Impact on existing workflows:** Workflows that accidentally used bare scalars instead of arrays will now run with a matrix context where they previously did not. The functional result is usually the same, but `matrix.*` variables are now populated. This is considered an improvement, but is a semantic change.

---

### 2. Unevaluated template expressions fall back to a literal string value

The actual resolution of `${{ fromJSON(needs.job1.outputs.versions) }}` into an array happens **before** `Matrix()` is called — in `EvaluateYamlNode` inside `pkg/runner/runner.go`. This PR does not change that evaluation path.

If evaluation succeeds → the YAML node already contains a proper array → `Matrix()` decodes it normally.

If evaluation **fails or is skipped** (e.g., expression not yet resolved) → the scalar string is now wrapped as `["${{ fromJSON(...) }}"]` → the job runs **one iteration with the literal unexpanded string** as the matrix value.

> **Note:** The test `TestJobMatrix/matrix_with_template_expression` covers this fallback path, not the successful resolution path. The literal-wrap behavior is the fallback, not the primary mechanism for template expression support.

---

## 🔑 Key Changes

### 1. Flexible Matrix Decoding (`pkg/model/workflow.go`)

#### New Helper Function: `normalizeMatrixValue`
```go
func normalizeMatrixValue(key string, val interface{}) ([]interface{}, error)
```
- Converts a single matrix value to the expected array format
- Handles three cases:
  1. **Arrays**: Pass through unchanged
  2. **Scalars**: Wrap in single-element array (including template expressions)
  3. **Invalid types** (maps, etc.): Return error to detect misconfigurations early

**Supported scalar types:**
- `string` — including unevaluated template expressions
- `int`, `float64` — numeric values
- `bool` — boolean values
- `nil` — null values

#### Enhanced `Matrix()` Method

**Before:**
```go
func (j *Job) Matrix() map[string][]interface{} {
    if j.Strategy.RawMatrix.Kind == yaml.MappingNode {
        var val map[string][]interface{}
        if !decodeNode(j.Strategy.RawMatrix, &val) {
            return nil
        }
        return val
    }
    return nil
}
```

**After:**
```go
func (j *Job) Matrix() map[string][]interface{} {
    if j.Strategy == nil || j.Strategy.RawMatrix.Kind != yaml.MappingNode {
        return nil
    }

    // First decode to flexible map[string]interface{} to handle scalars and arrays
    var flexVal map[string]interface{}
    err := j.Strategy.RawMatrix.Decode(&flexVal)
    if err != nil {
        // If that fails, try the strict array format
        var val map[string][]interface{}
        if !decodeNode(j.Strategy.RawMatrix, &val) {
            return nil
        }
        return val
    }

    // Convert flexible format to expected format with validation
    val := make(map[string][]interface{})
    for k, v := range flexVal {
        normalized, err := normalizeMatrixValue(k, v)
        if err != nil {
            log.Errorf("matrix validation error: %v", err)
            return nil
        }
        val[k] = normalized
    }
    return val
}
```

**Key improvements:**
- Handles unevaluated template expressions as scalar strings
- Automatic conversion to array format for consistent processing
- Validation to catch nested maps and invalid configurations
- Fallback to strict array format decoding for backward compatibility
- Early nil check for strategy to prevent nil pointer panics

---

## 🎁 Use Cases

### Example 1: Template Expression in Matrix (primary motivation)
```yaml
jobs:
  setup:
    runs-on: ubuntu-latest
    outputs:
      versions: ${{ steps.version-map.outputs.versions }}
    steps:
      - id: version-map
        run: echo "versions=[\"1.17\",\"1.18\",\"1.19\"]" >> $GITHUB_OUTPUT

  test:
    needs: setup
    strategy:
      matrix:
        # EvaluateYamlNode resolves this to an array before Matrix() is called
        version: ${{ fromJSON(needs.setup.outputs.versions) }}
    runs-on: ubuntu-latest
    steps:
      - run: echo "Testing version ${{ matrix.version }}"
```

### Example 2: Mixed Scalar and Array Values
```yaml
jobs:
  build:
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest]  # Array — unchanged
        go-version: 1.19                      # Scalar — wrapped to [1.19]
        node: [14, 16, 18]                    # Array — unchanged
    runs-on: ${{ matrix.os }}
```

### Example 3: Single Value Matrix
```yaml
jobs:
  deploy:
    strategy:
      matrix:
        environment: production  # Scalar — wrapped to ["production"]
    runs-on: ubuntu-latest
```

---

##  Benefits

-  **Feature Parity with GitHub Actions**: Supports template expressions in matrix strategies
- 🔒 **Robust Validation**: Detects misconfigured matrices (nested maps) early
- 🔄 **Backward Compatible**: Existing array-based workflows continue to work
- 📝 **Clear Error Messages**: Helpful validation messages for debugging
- 🛡️ **Type Safety**: Handles all YAML scalar types correctly

---

## 🚀 Breaking Changes

**Technically none in a strictly backward-incompatible sense**, but two semantic changes exist (see [⚠️ Semantic Changes](#️-semantic-changes-read-before-merging) above):

1. Scalar matrix values now iterate (previously they caused a `nil` matrix / no iteration)
2. Unevaluated template strings now produce a literal iteration instead of skipping

Both changes are improvements in practice, but downstream users should be aware.

---

## 📚 Related Work

- Related Gitea Server PR: [go-gitea/gitea#36564](https://github.com/go-gitea/gitea/pull/36564)
- Enables dynamic matrix generation from job outputs
- Supports advanced CI/CD patterns used in GitHub Actions

## 🔗 References

- [GitHub Actions: Matrix strategies](https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs)
- [GitHub Actions: Expression syntax](https://docs.github.com/en/actions/learn-github-actions/expressions)
- [GitHub Actions: `fromJSON` in matrix context](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix)

Reviewed-on: https://gitea.com/gitea/act/pulls/149
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: Pascal Zimmermann <pascal.zimmermann@theiotstudio.com>
Co-committed-by: Pascal Zimmermann <pascal.zimmermann@theiotstudio.com>
This commit is contained in:
Pascal Zimmermann
2026-04-19 22:36:34 +00:00
committed by silverwind
parent f923badec7
commit 15dd63a839
4 changed files with 351 additions and 3 deletions

View File

@@ -21,6 +21,42 @@ Tags:
- `nektos/v0.10.1` -> `v0.1001.*`, not ~~`v0.101.*`~~
- `nektos/v0.3.100` -> not ~~`v0.3100.*`~~, I don't think it's really going to happen, if it does, we can find a way to handle it.
## Gitea-specific changes
### Matrix strategy: scalar values and template expressions
This fork extends the matrix strategy parser [workflow.go](pkg/model/workflow.go) to accept
bare scalar YAML values in addition to arrays, and to handle unevaluated template
expressions gracefully.
**Scalar wrapping**
A matrix key written without brackets is automatically promoted to a
single-element array:
```yaml
strategy:
matrix:
go-version: 1.21 # treated as [1.21]
os: ubuntu-latest # treated as ["ubuntu-latest"]
```
> [!NOTE]
> Previously such a value caused the matrix decoding to fail and the job ran *without*
> a matrix context (`matrix.*` variables were undefined). Now the job runs *one* matrix iteration with the scalar as the
> value. Existing workflows that used scalars by accident may see a difference in which matrix variables are populated.
**Template expression support (`${{ fromJSON(...) }}`)**
Template expressions in the matrix are resolved by `EvaluateYamlNode`
(`pkg/runner/runner.go`) *before* `Matrix()` is called. When successful, the
expression is replaced by a proper YAML sequence and the matrix expands
normally.
If the expression cannot be resolved (e.g., the necessary context is not yet
available), the literal string is wrapped as a one-element array, and the job
runs once with the unexpanded string as the matrix value (graceful degradation).
---
![act-logo](https://raw.githubusercontent.com/wiki/nektos/act/img/logo-150.png)