efforting.tech/gitea
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor integration tests infrastructure (#37462)

Browse Source 
Co-authored-by: Claude (Opus 4.7) <noreply@anthropic.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This commit is contained in:
silverwind
2026-04-29 18:37:38 +02:00
committed by GitHub
parent 9262803621
commit d57d06335d
26 changed files with 331 additions and 544 deletions
Download Patch File Download Diff File Expand all files Collapse all files

148
tests/integration/README.md
View File

@@ -1,20 +1,77 @@
# Integration tests
Integration tests can be run with make commands for the
appropriate backends, namely:
```shell
make test-sqlite
make test-pgsql
make test-mysql
make test-mssql
Integration tests can be run with command `make test-integration`.
Environment variable `GITEA_TEST_DATABASE` can be used to specify the database type for testing.
If you encounter some errors like mismatched database version, SSH push errors, etc.,
you can try to perform a clean build by: `make clean build`.
## Run sqlite integration tests
Start tests directly (empty `GITEA_TEST_DATABASE` defaults to sqlite):
```
make test-integration
```
Make sure to perform a clean build before running tests:
## Run MySQL integration tests
Set up a MySQL database inside docker:
```
make clean build
docker run -e "MYSQL_DATABASE=test" -e "MYSQL_ALLOW_EMPTY_PASSWORD=yes" -p 3306:3306 --rm --name mysql mysql:latest #(just ctrl-c to stop db and clean the container)
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --rm --name elasticsearch elasticsearch:7.6.0 #(in a second terminal, just ctrl-c to stop db and clean the container)
```
## Run tests via local act_runner
Start tests based on the database container:
```
GITEA_TEST_DATABASE=mysql TEST_MYSQL_HOST=localhost:3306 TEST_MYSQL_DBNAME=test TEST_MYSQL_USERNAME=root TEST_MYSQL_PASSWORD='' make test-integration
```
## Run pgsql integration tests
Set up a pgsql database inside docker:
```
docker run -e "POSTGRES_DB=test" -e "POSTGRES_USER=postgres" -e "POSTGRES_PASSWORD=postgres" -p 5432:5432 --rm --name pgsql postgres:latest #(just ctrl-c to stop db and clean the container)
```
Set up minio inside docker:
```
docker run --rm -p 9000:9000 -e MINIO_ROOT_USER=123456 -e MINIO_ROOT_PASSWORD=12345678 --name minio bitnamilegacy/minio:2023.8.31
```
Start tests based on the database container:
```
GITEA_TEST_DATABASE=pgsql TEST_MINIO_ENDPOINT=localhost:9000 TEST_PGSQL_HOST=localhost:5432 TEST_PGSQL_DBNAME=postgres TEST_PGSQL_USERNAME=postgres TEST_PGSQL_PASSWORD=postgres make test-integration
```
## Run mssql integration tests
Set up a mssql database inside docker:
```
docker run -e "ACCEPT_EULA=Y" -e "MSSQL_PID=Standard" -e "SA_PASSWORD=MwantsaSecurePassword1" -p 1433:1433 --rm --name mssql microsoft/mssql-server-linux:latest #(just ctrl-c to stop db and clean the container)
```
Start tests based on the database container:
```
GITEA_TEST_DATABASE=mssql TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=gitea_test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-integration
```
## Running individual tests
Example command to run GPG test:
```
GITEA_TEST_DATABASE=... make test-integration#GPG
```
## Run Gitea Actions tests via local act_runner
### Run all jobs
@@ -22,7 +79,7 @@ make clean build
act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest
```
Warning: This file defines many jobs, so it will be resource-intensive and therefor not recommended.
Warning: This file defines many jobs, so it will be resource-intensive and therefore not recommended.
### Run single job
@@ -35,72 +92,3 @@ You can list all job names via:
```SHELL
act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest -l
```
## Run sqlite integration tests
Start tests
```
make test-sqlite
```
## Run MySQL integration tests
Setup a MySQL database inside docker
```
docker run -e "MYSQL_DATABASE=test" -e "MYSQL_ALLOW_EMPTY_PASSWORD=yes" -p 3306:3306 --rm --name mysql mysql:latest #(just ctrl-c to stop db and clean the container)
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --rm --name elasticsearch elasticsearch:7.6.0 #(in a second terminal, just ctrl-c to stop db and clean the container)
```
Start tests based on the database container
```
TEST_MYSQL_HOST=localhost:3306 TEST_MYSQL_DBNAME=test TEST_MYSQL_USERNAME=root TEST_MYSQL_PASSWORD='' make test-mysql
```
## Run pgsql integration tests
Setup a pgsql database inside docker
```
docker run -e "POSTGRES_DB=test" -e "POSTGRES_USER=postgres" -e "POSTGRES_PASSWORD=postgres" -p 5432:5432 --rm --name pgsql postgres:latest #(just ctrl-c to stop db and clean the container)
```
Setup minio inside docker
```
docker run --rm -p 9000:9000 -e MINIO_ROOT_USER=123456 -e MINIO_ROOT_PASSWORD=12345678 --name minio bitnamilegacy/minio:2023.8.31
```
Start tests based on the database container
```
TEST_MINIO_ENDPOINT=localhost:9000 TEST_PGSQL_HOST=localhost:5432 TEST_PGSQL_DBNAME=postgres TEST_PGSQL_USERNAME=postgres TEST_PGSQL_PASSWORD=postgres make test-pgsql
```
## Run mssql integration tests
Setup a mssql database inside docker
```
docker run -e "ACCEPT_EULA=Y" -e "MSSQL_PID=Standard" -e "SA_PASSWORD=MwantsaSecurePassword1" -p 1433:1433 --rm --name mssql microsoft/mssql-server-linux:latest #(just ctrl-c to stop db and clean the container)
```
Start tests based on the database container
```
TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=gitea_test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql
```
## Running individual tests
Example command to run GPG test:
For SQLite:
```
make test-sqlite#GPG
```
For other databases(replace `mssql` to `mysql`, or `pgsql`):
```
TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql#GPG
```
## Setting timeouts for declaring long-tests and long-flushes
We appreciate that some testing machines may not be very powerful and
the default timeouts for declaring a slow test or a slow clean-up flush
may not be appropriate.
You can set the following environment variables:
```bash
GITEA_TEST_SLOW_RUN="10s" GITEA_TEST_SLOW_FLUSH="1s" make test-sqlite
```

91
tests/integration/README_ZH.md
View File

@@ -1,91 +0,0 @@
# 关于集成测试
使用如下 make 命令可以运行指定的集成测试：
```shell
make test-mysql
make test-pgsql
make test-sqlite
```
在执行集成测试命令前请确保清理了之前的构建环境，清理命令如下：
```
make clean build
```
## 如何在本地 act_runner 上运行测试
### 运行所有任务
```
act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest
```
警告:由于在此文件中定义了许多任务，因此此操作将花费太多的CPU和内存来运行。所以不建议这样做。
### 运行单个任务
```SHELL
act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest -j <job_name>
```
您可以通过以下方式列出所有任务名称:
```SHELL
act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest -l
```
## 如何使用 sqlite 数据库进行集成测试
使用该命令执行集成测试
```
make test-sqlite
```
## 如何使用 mysql 数据库进行集成测试
首先在docker容器里部署一个 mysql 数据库
```
docker run -e "MYSQL_DATABASE=test" -e "MYSQL_ALLOW_EMPTY_PASSWORD=yes" -p 3306:3306 --rm --name mysql mysql:8 #(just ctrl-c to stop db and clean the container)
```
之后便可以基于这个数据库进行集成测试
```
TEST_MYSQL_HOST=localhost:3306 TEST_MYSQL_DBNAME=test TEST_MYSQL_USERNAME=root TEST_MYSQL_PASSWORD='' make test-mysql
```
## 如何使用 pgsql 数据库进行集成测试
同上，首先在 docker 容器里部署一个 pgsql 数据库
```
docker run -e "POSTGRES_DB=test" -e "POSTGRES_USER=postgres" -e "POSTGRES_PASSWORD=postgres" -p 5432:5432 --rm --name pgsql postgres:latest #(just ctrl-c to stop db and clean the container)
```
在docker内设置minio
```
docker run --rm -p 9000:9000 -e MINIO_ROOT_USER=123456 -e MINIO_ROOT_PASSWORD=12345678 --name minio bitnamilegacy/minio:2023.8.31
```
之后便可以基于这个数据库进行集成测试
```
TEST_MINIO_ENDPOINT=localhost:9000 TEST_PGSQL_HOST=localhost:5432 TEST_PGSQL_DBNAME=postgres TEST_PGSQL_USERNAME=postgres TEST_PGSQL_PASSWORD=postgres make test-pgsql
```
## Run mssql integration tests
同上，首先在 docker 容器里部署一个 mssql 数据库
```
docker run -e "ACCEPT_EULA=Y" -e "MSSQL_PID=Standard" -e "SA_PASSWORD=MwantsaSecurePassword1" -p 1433:1433 --rm --name mssql microsoft/mssql-server-linux:latest #(just ctrl-c to stop db and clean the container)
```
之后便可以基于这个数据库进行集成测试
```
TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=gitea_test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql
```
## 如何进行自定义的集成测试
下面的示例展示了怎样在集成测试中只进行 GPG 测试：
sqlite 数据库:
```
make test-sqlite#GPG
```
其它数据库(把 MSSQL 替换为 MYSQL, PGSQL):
```
TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql#GPG
```

6
tests/integration/api_org_avatar_test.go
View File

@@ -7,9 +7,11 @@ import (
"encoding/base64"
"net/http"
"os"
"path/filepath"
"testing"
auth_model "code.gitea.io/gitea/models/auth"
"code.gitea.io/gitea/modules/setting"
api "code.gitea.io/gitea/modules/structs"
"code.gitea.io/gitea/tests"
@@ -24,7 +26,7 @@ func TestAPIUpdateOrgAvatar(t *testing.T) {
token := getTokenForLoggedInUser(t, session, auth_model.AccessTokenScopeWriteOrganization)
// Test what happens if you use a valid image
avatar, err := os.ReadFile("tests/integration/avatar.png")
avatar, err := os.ReadFile(filepath.Join(setting.GetGiteaTestSourceRoot(), "tests/integration/avatar.png"))
assert.NoError(t, err)
if err != nil {
assert.FailNow(t, "Unable to open avatar.png")
@@ -48,7 +50,7 @@ func TestAPIUpdateOrgAvatar(t *testing.T) {
MakeRequest(t, req, http.StatusBadRequest)
// Test what happens if you use a file that is not an image
text, err := os.ReadFile("tests/integration/README.md")
text, err := os.ReadFile(filepath.Join(setting.GetGiteaTestSourceRoot(), "tests/integration/README.md"))
assert.NoError(t, err)
if err != nil {
assert.FailNow(t, "Unable to open README.md")

6
tests/integration/api_repo_avatar_test.go
View File

@@ -8,12 +8,14 @@ import (
"fmt"
"net/http"
"os"
"path/filepath"
"testing"
auth_model "code.gitea.io/gitea/models/auth"
repo_model "code.gitea.io/gitea/models/repo"
"code.gitea.io/gitea/models/unittest"
user_model "code.gitea.io/gitea/models/user"
"code.gitea.io/gitea/modules/setting"
api "code.gitea.io/gitea/modules/structs"
"code.gitea.io/gitea/tests"
@@ -28,7 +30,7 @@ func TestAPIUpdateRepoAvatar(t *testing.T) {
token := getUserToken(t, user2.LowerName, auth_model.AccessTokenScopeWriteRepository)
// Test what happens if you use a valid image
avatar, err := os.ReadFile("tests/integration/avatar.png")
avatar, err := os.ReadFile(filepath.Join(setting.GetGiteaTestSourceRoot(), "tests/integration/avatar.png"))
assert.NoError(t, err)
if err != nil {
assert.FailNow(t, "Unable to open avatar.png")
@@ -52,7 +54,7 @@ func TestAPIUpdateRepoAvatar(t *testing.T) {
MakeRequest(t, req, http.StatusBadRequest)
// Test what happens if you use a file that is not an image
text, err := os.ReadFile("tests/integration/README.md")
text, err := os.ReadFile(filepath.Join(setting.GetGiteaTestSourceRoot(), "tests/integration/README.md"))
assert.NoError(t, err)
if err != nil {
assert.FailNow(t, "Unable to open README.md")

29
tests/integration/api_repo_file_create_test.go
View File

@@ -124,35 +124,6 @@ func getExpectedFileResponseForCreate(info apiFileResponseInfo) *api.FileRespons
return ret
}
func BenchmarkAPICreateFileSmall(b *testing.B) {
onGiteaRun(b, func(b *testing.B, u *url.URL) {
user2 := unittest.AssertExistsAndLoadBean(b, &user_model.User{ID: 2}) // owner of the repo1 & repo16
repo1 := unittest.AssertExistsAndLoadBean(b, &repo_model.Repository{ID: 1}) // public repo
b.ResetTimer()
for n := 0; b.Loop(); n++ {
treePath := fmt.Sprintf("update/file%d.txt", n)
_, _ = createFile(user2, repo1, treePath)
}
})
}
func BenchmarkAPICreateFileMedium(b *testing.B) {
data := make([]byte, 10*1024*1024)
onGiteaRun(b, func(b *testing.B, u *url.URL) {
user2 := unittest.AssertExistsAndLoadBean(b, &user_model.User{ID: 2}) // owner of the repo1 & repo16
repo1 := unittest.AssertExistsAndLoadBean(b, &repo_model.Repository{ID: 1}) // public repo
b.ResetTimer()
for n := 0; b.Loop(); n++ {
treePath := fmt.Sprintf("update/file%d.txt", n)
copy(data, treePath)
_, _ = createFile(user2, repo1, treePath)
}
})
}
func TestAPICreateFile(t *testing.T) {
onGiteaRun(t, func(t *testing.T, u *url.URL) {
user2 := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 2}) // owner of the repo1 & repo16

6
tests/integration/api_user_avatar_test.go
View File

@@ -7,9 +7,11 @@ import (
"encoding/base64"
"net/http"
"os"
"path/filepath"
"testing"
auth_model "code.gitea.io/gitea/models/auth"
"code.gitea.io/gitea/modules/setting"
api "code.gitea.io/gitea/modules/structs"
"code.gitea.io/gitea/tests"
@@ -24,7 +26,7 @@ func TestAPIUpdateUserAvatar(t *testing.T) {
token := getTokenForLoggedInUser(t, session, auth_model.AccessTokenScopeWriteUser)
// Test what happens if you use a valid image
avatar, err := os.ReadFile("tests/integration/avatar.png")
avatar, err := os.ReadFile(filepath.Join(setting.GetGiteaTestSourceRoot(), "tests/integration/avatar.png"))
assert.NoError(t, err)
if err != nil {
assert.FailNow(t, "Unable to open avatar.png")
@@ -48,7 +50,7 @@ func TestAPIUpdateUserAvatar(t *testing.T) {
MakeRequest(t, req, http.StatusBadRequest)
// Test what happens if you use a file that is not an image
text, err := os.ReadFile("tests/integration/README.md")
text, err := os.ReadFile(filepath.Join(setting.GetGiteaTestSourceRoot(), "tests/integration/README.md"))
assert.NoError(t, err)
if err != nil {
assert.FailNow(t, "Unable to open README.md")

6
tests/integration/gpg_ssh_git_test.go
View File

@@ -12,6 +12,7 @@ import (
"net/http"
"net/url"
"os"
"path/filepath"
"testing"
auth_model "code.gitea.io/gitea/models/auth"
@@ -325,10 +326,11 @@ func crudActionCreateFile(_ *testing.T, ctx APITestContext, user *user_model.Use
}
func importTestingKey() (*openpgp.Entity, error) {
if _, _, err := process.GetManager().Exec("gpg --import tests/integration/private-testing.key", "gpg", "--import", "tests/integration/private-testing.key"); err != nil {
keyPath := filepath.Join(setting.GetGiteaTestSourceRoot(), "tests/integration/private-testing.key")
if _, _, err := process.GetManager().Exec("gpg --import "+keyPath, "gpg", "--import", keyPath); err != nil {
return nil, err
}
keyringFile, err := os.Open("tests/integration/private-testing.key")
keyringFile, err := os.Open(keyPath)
if err != nil {
return nil, err
}

2
tests/integration/integration_test.go
View File

@@ -87,7 +87,7 @@ func testMain(m *testing.M) int {
graceful.InitManager(managerCtx)
defer cancel()
err := tests.InitTest()
err := tests.InitIntegrationTest()
if err != nil {
return testlogger.MainErrorf("InitTest error: %v", err)
}

11
tests/integration/migration-test/migration_test.go
View File

@@ -37,6 +37,7 @@ var currentEngine *xorm.Engine
func initMigrationTest(t *testing.T) func() {
testlogger.Init()
require.NoError(t, setting.PrepareIntegrationTestConfig())
setting.SetupGiteaTestEnv()
assert.NotEmpty(t, setting.RepoRootPath)
@@ -49,12 +50,12 @@ func initMigrationTest(t *testing.T) func() {
}
func availableVersions() ([]string, error) {
migrationsDir, err := os.Open("tests/integration/migration-test")
migrationsDir, err := os.Open(filepath.Join(setting.GetGiteaTestSourceRoot(), "tests/integration/migration-test"))
if err != nil {
return nil, err
}
defer migrationsDir.Close()
versionRE, err := regexp.Compile("gitea-v(?P<version>.+)\\." + regexp.QuoteMeta(setting.Database.Type.String()) + "\\.sql.gz")
versionRE, err := regexp.Compile("gitea-v(?P<version>.+)" + regexp.QuoteMeta("."+setting.Database.Type.String()+".sql.gz"))
if err != nil {
return nil, err
}
@@ -75,7 +76,7 @@ func availableVersions() ([]string, error) {
}
func readSQLFromFile(version string) (string, error) {
filename := fmt.Sprintf("tests/integration/migration-test/gitea-v%s.%s.sql.gz", version, setting.Database.Type)
filename := filepath.Join(setting.GetGiteaTestSourceRoot(), "tests/integration/migration-test", fmt.Sprintf("gitea-v%s.%s.sql.gz", version, setting.Database.Type))
if _, err := os.Stat(filename); os.IsNotExist(err) {
return "", nil
@@ -97,7 +98,7 @@ func readSQLFromFile(version string) (string, error) {
if err != nil {
return "", err
}
return string(bytes.TrimPrefix(buf, []byte{'\xef', '\xbb', '\xbf'})), nil
return string(bytes.TrimPrefix(buf, []byte("\xef\xbb\xbf"))), nil
}
func restoreOldDB(t *testing.T, version string) {
@@ -230,6 +231,8 @@ func restoreOldDB(t *testing.T, version string) {
assert.NoError(t, err, "Failure whilst running: %s\nError: %v", statement, err)
}
db.Close()
default:
assert.Failf(t, "unsupported database type", "setting.Database.Type=%v", setting.Database.Type)
}
}

2
tests/mssql.ini.tmpl
View File

@@ -1,4 +1,4 @@
WORK_PATH = {{WORK_PATH}}
WORK_PATH = {{TEST_WORK_PATH}}
APP_NAME = Gitea: Git with a cup of tea
RUN_MODE = prod

2
tests/mysql.ini.tmpl
View File

@@ -1,4 +1,4 @@
WORK_PATH = {{WORK_PATH}}
WORK_PATH = {{TEST_WORK_PATH}}
APP_NAME = Gitea: Git with a cup of tea
RUN_MODE = prod

2
tests/pgsql.ini.tmpl
View File

@@ -1,4 +1,4 @@
WORK_PATH = {{WORK_PATH}}
WORK_PATH = {{TEST_WORK_PATH}}
APP_NAME = Gitea: Git with a cup of tea
RUN_MODE = prod

6
tests/sqlite.ini.tmpl
View File

@@ -1,4 +1,4 @@
WORK_PATH = {{WORK_PATH}}
WORK_PATH = {{TEST_WORK_PATH}}
APP_NAME = Gitea: Git with a cup of tea
RUN_MODE = prod
@@ -88,7 +88,7 @@ ENABLED = true
[markup.html]
ENABLED = true
FILE_EXTENSIONS = .html
RENDER_COMMAND = go run tools/test-echo.go
RENDER_COMMAND = go run {{GITEA_TEST_ROOT}}/tools/test-echo.go
;RENDER_COMMAND = cat
;IS_INPUT_FILE = true
RENDER_CONTENT_MODE = sanitized
@@ -96,7 +96,7 @@ RENDER_CONTENT_MODE = sanitized
[markup.no-sanitizer]
ENABLED = true
FILE_EXTENSIONS = .no-sanitizer
RENDER_COMMAND = go run tools/test-echo.go
RENDER_COMMAND = go run {{GITEA_TEST_ROOT}}/tools/test-echo.go
; This test case is reused, at first it is used to test "no-sanitizer" (sandbox doesn't take effect here)
; Then it will be updated and used to test "iframe + sandbox-disabled"
RENDER_CONTENT_MODE = no-sanitizer

17
tests/test_utils.go
View File

@@ -6,7 +6,6 @@ package tests
import (
"database/sql"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
@@ -25,17 +24,14 @@ import (
"github.com/stretchr/testify/assert"
)
func InitTest() error {
func InitIntegrationTest() error {
testlogger.Init()
if os.Getenv("GITEA_TEST_CONF") == "" {
// By default, use sqlite.ini for testing, then IDE like GoLand can start the test process with debugger.
// It's easier for developers to debug bugs step by step with a debugger.
// Notice: when doing "ssh push", Gitea executes sub processes, debugger won't work for the sub processes.
giteaConf := "tests/sqlite.ini"
_ = os.Setenv("GITEA_TEST_CONF", giteaConf)
_, _ = fmt.Fprintf(os.Stderr, "Environment variable GITEA_TEST_CONF not set - defaulting to %s\n", giteaConf)
err := setting.PrepareIntegrationTestConfig()
if err != nil {
return err
}
setting.SetupGiteaTestEnv()
setting.Repository.DefaultBranch = "master" // many test code still assume that default branch is called "master"
@@ -132,6 +128,9 @@ func InitTest() error {
return err
}
}
case setting.Database.Type.IsSQLite3():
default:
return fmt.Errorf("unsupported database type: %s", setting.Database.Type)
}
routers.InitWebInstalled(graceful.GetManager().HammerContext())