学习 Dapr 的构建
Dapr构建学习
- 1: kit仓库的构建学习
- 1.1: 构建简介
- 1.2: Makefile文件
- 1.3: codedov文件
- 1.4: workflow
- 2: components-contrib仓库的构建学习
- 2.1: component-contrib仓库构建简介
- 2.2: Makefile文件
- 2.3: codedov文件
- 2.4: components-contrib仓库的workflow
- 2.4.1: components-contrib.yml
- 3: dapr仓库的构建学习
- 3.1: 构建简介
- 3.2: Makefile文件
- 3.3: Makefile codegen部分
- 3.4: Makefile docker部分
- 3.5: 测试相关的构建学习
- 3.5.1: Makefile tests部分
- 3.5.2: 端到端测试部分
- 3.5.3: 性能测试部分
- 4: quickstarts仓库的构建学习
- 4.1: quickstarts 简介
- 4.2: quickstarts 构建
- 5: docs仓库的构建学习
- 5.1: 安装配置hugo
- 5.2: 构建daprdocs的hugo文档
- 6: java-sdk仓库的构建学习
- 6.1: validate脚本
1.1 - 构建简介
kit仓库适合入门
kit仓库是代码量最小的仓库，也是构建最简单的仓库，适合入门。
备注：后续其他仓库的构建学习中，不会重复kit仓库中已有的内容，因此，推荐先学习kit仓库。
1.2 - Makefile文件
定义和获取变量，定义 test/lint/go.mod/check-diff等target
https://github.com/dapr/kit/blob/main/Makefile
变量定义
先是一堆变量定义。
golang变量
export GO111MODULE ?= on
export GOPROXY ?= https://proxy.golang.org
export GOSUMDB ?= sum.golang.org
# By default, disable CGO_ENABLED. See the details on https://golang.org/cmd/cgo
CGO ?= 0
# GOARCH 和 GOOS 的定义位置在后面一些
export GOARCH ?= $(TARGET_ARCH_LOCAL)
export GOOS ?= $(TARGET_OS_LOCAL)
“?=” 是 Makefile 中的赋值符号，语义是：如果之前没有赋值就赋予等号后面的值，如果之前有赋值就忽略。等价于：
if (GO111MODULE == null || GO111MODULE.length() == 0) {
GO111MODULE = "on"
}
GOPROXY 容易理解，为了加速我们一般设置为国内的代理，典型如 goproxy.cn。构建中如果发现下载go package的速度不理想，可以通过设置 goproxy 来优化。
GO111MODULE 如果没有特别设置，则 dapr 会要求设置为 on。关于 GO111MODULE 这个设置的详细介绍，请参考：
GOSUMDB 是为了保证开发者的依赖库不被人恶意劫持篡改，Go 会自动去这个服务器进行数据校验，这里取默认值就好，或者为了优化在国内的速度，可以设置为 sum.golang.google.cn。参考：
CGO 默认是关闭的。
git变量
定义了两个git相关的变量：
GIT_COMMIT = $(shell git rev-list -1 HEAD)
GIT_VERSION = $(shell git describe --always --abbrev=7 --dirty)
这两行代码相当于执行了下面两条git命令，然后将结果赋值给了 GIT_COMMIT 和 GIT_VERSION 变量：
$ git rev-list -1 HEAD
29c3d7dee532e510de7adc31bc43a21535b5e603
$ git describe --always --abbrev=7 --dirty
29c3d7d
对比 git log 命令的输入：
$ git log
commit 29c3d7dee532e510de7adc31bc43a21535b5e603 (HEAD -> main, origin/main, origin/HEAD)
Author: Will <witsai@microsoft.com>
Date: Thu Dec 30 07:09:28 2021 -0800
......
commit 867d7d9f3e6454864b4357941bab7601ae1cbd0a
Author: Dmitry Shmulevich <dmitry.shmulevich@gmail.com>
Date: Thu Dec 9 12:03:02 2021 -0800
......
commit db8c68dfecc9515c6da148e9439730719f838d67
Author: greenie-msft <56556602+greenie-msft@users.noreply.github.com>
Date: Wed Nov 24 10:40:17 2021 -080
......
GIT_COMMIT 和 GIT_VERSION 拿到的是最新的一次 git commit 记录，只是 GIT_COMMIT 是完整的 commit，而 GIT_VERSION 是 commit 的前8个字符。
操作系统变量
通过 uname -m 命令来 LOCAL_ARCH 变量，然后判断一下，设置 TARGET_ARCH_LOCAL 变量：
LOCAL_ARCH := $(shell uname -m)
ifeq ($(LOCAL_ARCH),x86_64)
TARGET_ARCH_LOCAL=amd64
else ifeq ($(shell echo $(LOCAL_ARCH) | head -c 5),armv8)
TARGET_ARCH_LOCAL=arm64
else ifeq ($(shell echo $(LOCAL_ARCH) | head -c 4),armv)
TARGET_ARCH_LOCAL=arm
else
TARGET_ARCH_LOCAL=amd64
endif
然后是类似的获取 TARGET_OS_LOCAL 变量：
LOCAL_OS := $(shell uname)
ifeq ($(LOCAL_OS),Linux)
TARGET_OS_LOCAL = linux
else ifeq ($(LOCAL_OS),Darwin)
TARGET_OS_LOCAL = darwin
else
TARGET_OS_LOCAL ?= windows
endif
在得到 TARGET_ARCH_LOCAL 和 TARGET_OS_LOCAL 之后，就可以设置 GOARCH 和 GOOS 了：
export GOARCH ?= $(TARGET_ARCH_LOCAL)
export GOOS ?= $(TARGET_OS_LOCAL)
一些额外设置，对于windows系统会有所不同:
ifeq ($(GOOS),windows)
BINARY_EXT_LOCAL:=.exe
GOLANGCI_LINT:=golangci-lint.exe
# Workaround for https://github.com/golang/go/issues/40795
BUILDMODE:=-buildmode=exe
else
BINARY_EXT_LOCAL:=
GOLANGCI_LINT:=golangci-lint
endif
build target
然后就是正式的 build target 定义了。
Target: test
################################################################################
# Target: test #
################################################################################
.PHONY: test
test:
go test ./... $(COVERAGE_OPTS) $(BUILDMODE)
test target 执行测试，实际调用 go test，执行结果如下：
$ make test
go test ./...
ok github.com/dapr/kit/config 0.008s
ok github.com/dapr/kit/logger 0.007s
ok github.com/dapr/kit/retry 0.006s
$ make test
go test ./...
ok github.com/dapr/kit/config (cached)
ok github.com/dapr/kit/logger (cached)
ok github.com/dapr/kit/retry (cached)
注意第二次时会有cache，如果代码和测试代码都没有修改则不会真正的跑测试，而是给出上次缓存的结果。如果要强制重新跑全部的测试，可以先做一下clean：
go clean --testcache
Target: lint
################################################################################
# Target: lint #
################################################################################
.PHONY: lint
lint:
# Due to https://github.com/golangci/golangci-lint/issues/580, we need to add --fix for windows
$(GOLANGCI_LINT) run --timeout=20m
在本地执行 lint target之前，先要确认已经安装了 golangci-lint ，如果没有安装则会报错：
make lint
# Due to https://github.com/golangci/golangci-lint/issues/580, we need to add --fix for windows
golangci-lint run --timeout=20m
make: golangci-lint: Command not found
make: *** [Makefile:72: lint] Error 127
安装方式参考 https://golangci-lint.run/usage/install/ 。linux下执行如下命令：
$ curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.43.0
golangci/golangci-lint info checking GitHub for tag 'v1.43.0'
golangci/golangci-lint info found version: 1.43.0 for v1.43.0/linux/amd64
golangci/golangci-lint info installed /home/sky/work/soft/gopath/bin/golangci-lint
$ golangci-lint --version
golangci-lint has version 1.43.0 built from 861262b7 on 2021-11-03T11:57:46Z
再次执行 make lint 进行检查，发现一堆输出：
make lint
# Due to https://github.com/golangci/golangci-lint/issues/580, we need to add --fix for windows
golangci-lint run --timeout=20m
WARN [runner] The linter 'golint' is deprecated (since v1.41.0) due to: The repository of the linter has been archived by the owner. Replaced by revive.
config/decode.go:107:21 godot Comment should end in a period
config/decode.go:112:27 godot Comment should end in a period
config/decode.go:117:27 godot Comment should end in a period
logger/dapr_logger.go:31:17 revive var-declaration: should omit type string from declaration of var DaprVersion; it will be inferred from the right-hand side
logger/dapr_logger.go:71:16 exhaustivestruct DisableTimestamp, DisableHTMLEscape, DataKey, CallerPrettyfier, PrettyPrint are missing in JSONFormatter
logger/dapr_logger.go:76:16 exhaustivestruct ForceColors, DisableColors, ForceQuote, DisableQuote, EnvironmentOverrideColors, DisableTimestamp, FullTimestamp, DisableSorting, SortingFunc, DisableLevelTruncation, PadLevelText, QuoteEmptyFields, CallerPrettyfier are missing in TextFormatter
config/decode.go:86:3 forcetypeassert type assertion must be checked
config/decode.go:89:3 forcetypeassert type assertion must be checked
检查对比了一下 dapr CI 中是怎么进行 lint 检测的，发现原来 dapr ci 中用的是 golangci-lint ‘v1.31’ 版本：
Requested golangci-lint 'v1.31', using 'v1.31.0', calculation took 119ms
Installing golangci-lint v1.31.0...
Downloading https://github.com/golangci/golangci-lint/releases/download/v1.31.0/golangci-lint-1.31.0-linux-amd64.tar.gz ...
/usr/bin/tar xz --warning=no-unknown-keyword -C /home/runner -f /home/runner/work/_temp/0f22eea4-f347-44ce-af0e-a2575ae885ef
Installed golangci-lint into /home/runner/golangci-lint-1.31.0-linux-amd64/golangci-lint in 1106ms
删除安装的1.43版本，重新安装1.31版本：
$ rm /home/sky/work/soft/gopath/bin/golangci-lint
$ curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.31.0
golangci/golangci-lint info checking GitHub for tag 'v1.31.0'
golangci/golangci-lint info found version: 1.31.0 for v1.31.0/linux/amd64
golangci/golangci-lint info installed /home/sky/work/soft/gopath/bin/golangci-lint
$ golangci-lint --version
golangci-lint has version 1.31.0 built from 3d6d0e7 on 2020-09-07T15:14:41Z
再次执行，这次结果就和 dapr ci 对应上了。
$ make lint
# Due to https://github.com/golangci/golangci-lint/issues/580, we need to add --fix for windows
golangci-lint run --timeout=20m
切记
golangci-lint 一定要安装 1.31 版本！In m1 macbook
在 m1 macbook 上， 由于 1.31 版本发布较早，没有提供对 m1 （也就是darwin-arm64）的支持，因此上面的脚本在运行时并不能自动下载安装：
$ curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.31.0
golangci/golangci-lint info checking GitHub for tag 'v1.31.0'
golangci/golangci-lint info found version: 1.31.0 for v1.31.0/darwin/arm64
解决方法就是开启 Rosetta 来兼容 intel 芯片：
If you need to install Rosetta on your Mac - Apple Support
通常在第一次运行基于inte芯片构建的应用时会提示。
然后手工下载 v1.31.0 的 darwin-amd64 二进制文件：
- https://github.com/golangci/golangci-lint/releases/tag/v1.31.0
- https://github.com/golangci/golangci-lint/releases/download/v1.31.0/golangci-lint-1.31.0-darwin-amd64.tar.gz
将解压缩得到的 golangci-lint 文件移动到 gopath 下的 bin 目录即可。
Target: go.mod
################################################################################
# Target: go.mod #
################################################################################
.PHONY: go.mod
go.mod:
go mod tidy
这个好像没啥好说的，我个人其实更习惯直接执行 go mod tidy 命令。
Target: check-diff
################################################################################
# Target: check-diff #
################################################################################
.PHONY: check-diff
check-diff:
git diff --exit-code ./go.mod # check no changes
调用 git diff 命令检查 go.mod 文件是否有改动。
1.3 - codedov文件
codedov的配置文件
codecov 背景知识
https://docs.codecov.com/docs/codecov-yaml
Codecov Yaml文件是唯一的配置点，为开发者提供了一个透明的、版本控制的文件来调整所有的Codecov设置。
参考手册：
https://docs.codecov.com/docs/codecovyml-reference
验证工具：
https://api.codecov.io/validate
kit仓库配置
https://github.com/dapr/kit/blob/main/.codecov.yaml
kit仓库的 .codecov.yaml 文件极其简单，只配置了 coverage.status 的少数属性：
coverage:
# Commit status https://docs.codecov.io/docs/commit-status are used
# to block PR based on coverage threshold.
status:
project:
default:
target: auto
threshold: 0%
patch:
default:
informational: true
参考 https://docs.codecov.com/docs/commit-status 中的说明, status 检查的功能是:
Useful for blocking Pull Requests that don’t meet a particular coverage threshold.
有助于阻止不符合特定覆盖率阈值的Pull Requests。
target
取值可以是 auto 或者 number 。
选择一个最小的覆盖率，提交必须满足该覆盖率才能被认为是成功的。
auto将使用基础提交（PR请求基础或父提交）的覆盖率来进行比较。number可以指定一个精确的覆盖率目标，如75%或100%（接受字符串、int或float）。
threshold
取值为 number 。
允许覆盖率下降 X%，并发布成功状态。
informational
在信息模式下使用Codecov。默认为false。如果指定为 “true”，那么无论覆盖率是多少或其他设置是什么，结果都会通过。如果想在PR中向其他开发者公开codecov信息，而不一定要在这些信息上进行门禁，那么信息模式是非常好的选择。
本地执行
本地执行go test 命令，加上相关的参数之后就可以得到覆盖率信息：
$ go test ./... -coverprofile=coverage.txt -covermode=atomic
ok github.com/dapr/kit/config 0.008s coverage: 95.5% of statements
ok github.com/dapr/kit/logger 0.008s coverage: 87.0% of statements
ok github.com/dapr/kit/retry 0.006s coverage: 97.6% of statements
可以用工具查看更详细的输出：
go tool cover -html=coverage.txt
如果是使用IDE如 goland，可以参考：
https://www.jetbrains.com/help/go/code-coverage.html
最常见的方式是在编写单元测试案例时，右键 “More Run/Debug” -> “Run xxx.go.test with Coverage”，执行结束之后会自动打开 Coverage 视图。
CI执行
可以通过下面的地址查看：
https://app.codecov.io/gh/dapr/kit
CI执行 Coverage 检查的过程，请参见下一章 workflow 的详细说明。
1.4 - workflow
workflow
Dapr 采用 github action作为 CI，其中 kit 仓库的 workflow 是最简单的，只有一个 workflow：
https://github.com/dapr/kit/blob/main/.github/workflows/kit.yml
action基本设置
参考 github action 的说明：
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions
工作流(workflow) 是一个由一个或多个工作组成的可配置的自动化流程。你必须创建一个YAML文件来定义你的工作流配置。
工作流文件使用YAML语法，并且必须有一个
.yml或.yaml的文件扩展名。必须将工作流文件存储在版本库的
.github/workflows目录中。
kit仓库下 .github/workflows/kit.yml 文件的内容:
name: kit
on:
push:
branches:
- main
- release-*
tags:
- v*
pull_request:
branches:
- main
- release-*
......
name: 工作流的名称。GitHub 会在仓库的操作页面上显示工作流的名称。如果省略名称，GitHub 会将其设置为相对于仓库根目录的工作流文件路径。on: 要自动触发一个工作流程，使用 on 来定义哪些事件可以导致工作流程的运行。关于可用事件的列表，请参见 “触发工作流程的事件”。 可以定义单个或多个可以触发工作流程的事件，或设置一个时间计划。还可以限制工作流的执行，使其只发生在特定的文件、标记或分支变化上。这些选项将在以下章节中描述。on: push: 当推送到工作流仓库中时工作流将运行。on: pull_request: 当工作流仓库中有PR时工作流将运行。
push 和 pull_request 都带有过滤器配置：
on:
push:
branches:
- main
- release-*
tags:
- v*
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#using-filters
有些事件有过滤器，可以让你对工作流的运行时间有更多控制。
例如，推送事件有一个分支过滤器，它使你的工作流只在发生推送到符合分支过滤器的分支时运行，而不是在发生任何推送时运行。
jobs
https://docs.github.com/en/actions/using-jobs/using-jobs-in-a-workflow
工作流程的运行是由一个或多个作业组成的，默认情况下是平行运行的。要按顺序运行作业，你可以使用
job.<job_id>.needs关键字定义对其他作业的依赖性。每个作业都在由
runs-on指定的运行器环境中运行。
为了方便理解和对照，建议在 https://github.com/dapr/kit/actions 下打开任意一个实际运行过的 workflow，对照里面的输出进行学习。
matrix配置
dapr的build都是采用 matrix 方式配置，以便支持多个操作系统 (windows / linux / macos) 和架构(amd64和arm)。目前实际支持的是四个：
- linux arm
- linux amd64
- windows amd64
- darwin amd64
具体的配置内容是:
jobs:
build:
name: Build ${{ matrix.target_os }}_${{ matrix.target_arch }} binaries
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
target_arch: [arm, amd64]
include:
- os: ubuntu-latest
target_os: linux
- os: windows-latest
target_os: windows
- os: macOS-latest
target_os: darwin
exclude:
- os: windows-latest
target_arch: arm
- os: macOS-latest
target_arch: arm
三个操作系统和两个架构，本来是应该有6个组合的，但是通过 exclude 去掉了 windows + arm 和 macos + arm 两个，就剩下4个了。
对照 “name: Build ${{ matrix.target_os }}_${{ matrix.target_arch }} binaries” 这一行的配置和下图实际运行时的job就清晰了：
具体配置方法可参考：
https://docs.github.com/en/actions/using-jobs/using-a-build-matrix-for-your-jobs
go配置
这里配置有go相关的信息:
env:
GOVER: 1.16
GOOS: ${{ matrix.target_os }}
GOARCH: ${{ matrix.target_arch }}
GOPROXY: https://proxy.golang.org
GOLANGCI_LINT_VER: v1.31
其中 go 用的是 1.16 版本， 而 golangci-lint 是 v1.31 版本。
steps配置
kit.yml 中定义了6个step，分别是:
steps:
- name: Set up Go ${{ env.GOVER }}
- name: Check out code into the Go module directory
- name: Run golangci-lint
- name: Run make go.mod check-diff
- name: Run make test
- name: Codecov
下图是实际运行的 linux_amd64 job，可以看到它的 step 信息里面有和这6个steps对应的内容：
step: Set up job
这是 github action 自身的job，主要是准备操作系统和相关的软件。
其中linux采用的是ubuntu server 20.04.3：
windows 采用的是 windows server 2019:
macos 采用的是 11.6.1 :
以及准备相关的 actions，以备后续步骤使用：
Prepare workflow directory
Prepare all required actions
Getting action download info
Download action repository 'actions/setup-go@v1' (SHA:0caeaed6fd66a828038c2da3c0f662a42862658f)
Download action repository 'actions/checkout@v2' (SHA:ec3a7ce113134d7a93b817d10a8272cb61118579)
Download action repository 'golangci/golangci-lint-action@v2.2.1' (SHA:54a84d46fb7183443c869b1b7d0dc34f640fcfd7)
Download action repository 'codecov/codecov-action@v1' (SHA:29386c70ef20e286228c72b668a06fd0e8399192)
step: Set up Go
安装go的步骤，使用到前面准备的 action actions/setup-go@v1:
- name: Set up Go ${{ env.GOVER }}
uses: actions/setup-go@v1
with:
go-version: ${{ env.GOVER }}
运行时的实际日志为：
Run actions/setup-go@v1
with:
go-version: 1.16
env:
GOVER: 1.16
GOOS: linux
GOARCH: amd64
GOPROXY: https://proxy.golang.org
GOLANGCI_LINT_VER: v1.31
/usr/bin/tar xzC /home/runner/work/_temp/5958e440-fd50-4948-9fe2-e9948b31e2fc -f /home/runner/work/_temp/62766d82-8d66-40b2-a183-1c35bd69a430
step: Check out code
checkout 代码的步骤，使用到前面准备的 action actions/checkout@v2:
- name: Check out code into the Go module directory
uses: actions/checkout@v2
运行时的实际日志为：
Run actions/checkout@v2
with:
repository: dapr/kit
......
Syncing repository: dapr/kit
Deleting the contents of '/home/runner/work/kit/kit'
Initializing the repository
/usr/bin/git init /home/runner/work/kit/kit
......
Initialized empty Git repository in /home/runner/work/kit/kit/.git/
/usr/bin/git remote add origin https://github.com/dapr/kit
......
/usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +867d7d9f3e6454864b4357941bab7601ae1cbd0a:refs/remotes/origin/main
/usr/bin/git checkout --progress --force -B main refs/remotes/origin/main
/usr/bin/git log -1 --format='%H'
'867d7d9f3e6454864b4357941bab7601ae1cbd0a'
这个步骤比预料中要复杂：不是简单的 git clone 然后 git checkout branch/commit-id ，而是重新新建了一个空的git仓库，然后设置remote指向https://github.com/dapr/kit，在执行复杂的 fetch 和 checkout 命令。
step：Run golangci-lint
执行golangci-lint的步骤，使用到前面准备的 action golangci/golangci-lint-action@v2.2.1:
- name: Run golangci-lint
if: matrix.target_arch == 'amd64' && matrix.target_os == 'linux'
uses: golangci/golangci-lint-action@v2.2.1
with:
version: ${{ env.GOLANGCI_LINT_VER }}
运行时的实际日志为：
Run golangci/golangci-lint-action@v2.2.1
with:
version: v1.31
env:
GOVER: 1.16
GOOS: linux
GOARCH: amd64
GOPROXY: https://proxy.golang.org
GOLANGCI_LINT_VER: v1.31
GOROOT: /opt/hostedtoolcache/go/1.16.12/x64
Requested golangci-lint 'v1.31', using 'v1.31.0', calculation took 240ms
Installing golangci-lint v1.31.0...
Downloading https://github.com/golangci/golangci-lint/releases/download/v1.31.0/golangci-lint-1.31.0-linux-amd64.tar.gz ...
Cache not found for input keys: golangci-lint.cache-2710-0af295f28debd8dc18877aa104c7829fdda983ce, golangci-lint.cache-2710-, golangci-lint.cache-
/usr/bin/tar xz --warning=no-unknown-keyword -C /home/runner -f /home/runner/work/_temp/c61d00e3-ed1e-4299-acc1-480cd4b965b0
Installed golangci-lint into /home/runner/golangci-lint-1.31.0-linux-amd64/golangci-lint in 407ms
但比较有意思的是，在 golangci-lint-action 准备的过程中，没有使用前面构建好的go 1.16，而是自己从 cache 里面找到了一个 go 1.17.3 版本：
prepare environment
Finding needed golangci-lint version...
Setup go stable version spec 1
Found in cache @ /opt/hostedtoolcache/go/1.17.3/x64
Added go to the path
Successfully setup go version 1
go version go1.17.3 linux/amd64
go env
......
GOROOT="/opt/hostedtoolcache/go/1.17.3/x64"
GOTOOLDIR="/opt/hostedtoolcache/go/1.17.3/x64/pkg/tool/linux_amd64"
GOVERSION="go1.17.3"
最后执行 golangci-lint:
run golangci-lint
Running [/home/runner/golangci-lint-1.31.0-linux-amd64/golangci-lint run --out-format=github-actions] in [] ...
golangci-lint found no issues
Ran golangci-lint in 10405ms
step: Run make go.mod check-diff
执行 make go.mod 和 make check-diff 的步骤，使用到 Makefile 中定义的 target go.mod 和 check-diff:
- name: Run make go.mod check-diff
if: matrix.target_arch != 'arm' # arm 构架上还不用跑这个step
run: make go.mod check-diff
运行时的实际日志为：
Run make go.mod check-diff
make go.mod check-diff
shell: /usr/bin/bash -e {0}
env:
GOVER: 1.16
GOOS: linux
GOARCH: amd64
GOPROXY: https://proxy.golang.org
GOLANGCI_LINT_VER: v1.31
GOROOT: /opt/hostedtoolcache/go/1.17.3/x64 # 这里的goroot也乱了，用了 1.17.3
go mod tidy
go: downloading gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
git diff --exit-code ./go.mod # check no changes
step: Run make test
终于到了跑测试的step，用到了 Makefile 中定义的 target test ：
- name: Run make test
env:
COVERAGE_OPTS: "-coverprofile=coverage.txt -covermode=atomic"
if: matrix.target_arch != 'arm'
run: make test
对照 Makefile 中的 target test：
.PHONY: test
test:
go test ./... $(COVERAGE_OPTS) $(BUILDMODE)
kit.yml 这里设置了 COVERAGE_OPTS，内容为 “-coverprofile=coverage.txt -covermode=atomic”，因此以后本地跑 coverage 时也可以同样设置，以便本地执行的结果和CI中尽量一致。
运行时的实际日志为：
Run make test
go test ./... -coverprofile=coverage.txt -covermode=atomic
ok github.com/dapr/kit/config 0.014s coverage: 95.5% of statements
ok github.com/dapr/kit/logger 0.007s coverage: 87.0% of statements
ok github.com/dapr/kit/retry 0.004s coverage: 97.6% of statements
注意这里已经有了单元测试覆盖率的信息，我试了一下在本地开发环境中用同样的命令跑了一下，结果和CI的完全一致，因此以后可以在本地将覆盖率做好之后再提交PR，避免因为覆盖率问题反复提交：
go test ./... -coverprofile=coverage.txt -covermode=atomic
ok github.com/dapr/kit/config 0.008s coverage: 95.5% of statements
ok github.com/dapr/kit/logger 0.007s coverage: 87.0% of statements
ok github.com/dapr/kit/retry 0.006s coverage: 97.6% of statements
step: Codecov
执行覆盖率分析和上传的步骤，使用到前面准备的 action codecov/codecov-action@v1:
- name: Codecov
if: matrix.target_arch == 'amd64' && matrix.target_os == 'linux' # 目前是只在 amd 64 + linux 平台上跑覆盖率检查
uses: codecov/codecov-action@v1
运行时的实际日志为：
Run codecov/codecov-action@v1
/usr/bin/bash codecov.sh -n -F -Q github-action-v1.5.2
......
==> Searching for coverage reports in:
+ .
-> Found 2 reports
==> Detecting git/mercurial file structure
==> Reading reports
+ ./coverage.txt bytes=8253
+ ./.codecov.yaml bytes=260
Uploading reports
url: https://codecov.io
......
Reports have been successfully queued for processing at https://codecov.io/github/dapr/kit/commit/867d7d9f3e6454864b4357941bab7601ae1cbd0a
这里实际就是将上一个make test 步骤生成的覆盖率信息收集上传到 codecov.io。
step: Post Run golangci-lint
运行时的实际日志为：
Post job cleanup.
/usr/bin/tar --posix --use-compress-program zstd -T0 -cf cache.tzst -P -C /home/runner/work/kit/kit --files-from manifest.txt
Cache saved successfully
Saved cache for golangci-lint from paths '/home/runner/.cache/golangci-lint, /home/runner/.cache/go-build, /home/runner/go/pkg' in 795ms
打包了一下cache文件。
step: Post Check out code
没啥内容，忽略。
总结
由于 kit 仓库内容简单，因此它的CI流程和相应的 github action workflow 都非常简单，其核心内容就是执行下面三个检查:
- make test
- code coverage
- golangci-lint
其他步骤都是为这三个步骤做准备。
但麻雀虽小五脏俱全，github action workflow 相关的基本配置和基本流程都在。了解这些基本内容之后再继续看其他仓库的 workflow 就可以把重点放在其他地方。
2 - components-contrib仓库的构建学习
Dapr构建学习之components-contrib仓库
2.1 - component-contrib仓库构建简介
kit仓库适合入门
https://github.com/dapr/components-contrib
components-contrib 仓库相比 kit 仓库代码量要大很多仓库，但构建还是会比 dapr 仓库要简单。
备注：后续其他仓库的构建学习中，不会重复kit仓库中已有的内容。
2.2 - Makefile文件
定义和获取变量，定义 test/lint/go.mod/check-diff等target
https://github.com/dapr/components-contrib/blob/master/Makefile
变量定义
和 kit 仓库完全一致，跳过。
build target
然后就是正式的 build target 定义了。
其中 target test / target lint / target modtidy 和 kit 仓库完全一致，跳过。
Target: modtidy-all
在 modtidy 之外，还提供了一个 modtidy-all 的target，这是因为 conponents-contrib 仓库除了根目录下的 go.mod 之外，还在 ./tests/certification/ 目录下有多个 go.mod 文件。
find . -name go.mod
./go.mod
./tests/certification/bindings/azure/blobstorage/go.mod
./tests/certification/bindings/azure/cosmosdb/go.mod
./tests/certification/bindings/azure/servicebusqueues/go.mod
./tests/certification/go.mod
./tests/certification/pubsub/rabbitmq/go.mod
./tests/certification/pubsub/mqtt/go.mod
./tests/certification/pubsub/kafka/go.mod
./tests/certification/secretstores/azure/keyvault/go.mod
./tests/certification/state/sqlserver/go.mod
################################################################################
# Target: modtidy-all #
################################################################################
MODFILES := $(shell find . -name go.mod) # 列出所有的 go.mod 文件，列表如上面所示
define modtidy-target
.PHONY: modtidy-$(1)
modtidy-$(1):
# 进入每一个 go.mod 所在的目录，执行 go mod tidy，然后再返回原来的目录
cd $(shell dirname $(1)); go mod tidy -compat=1.17; cd -
endef
# Generate modtidy target action for each go.mod file
# 为每个go.mod文件生成modtidy target动作
$(foreach MODFILE,$(MODFILES),$(eval $(call modtidy-target,$(MODFILE))))
# Enumerate all generated modtidy targets
# Note that the order of execution matters: root and tests/certification go.mod
# are dependencies in each certification test. This order is preserved by the
# tree walk when finding the go.mod files.
TIDY_MODFILES:=$(foreach ITEM,$(MODFILES),modtidy-$(ITEM))
# Define modtidy-all action trigger to run make on all generated modtidy targets
.PHONY: modtidy-all
modtidy-all: $(TIDY_MODFILES)
实际执行的日志输出如下：
make modtidy-all
cd .; go mod tidy -compat=1.17; cd -
/home/sky/work/code/dapr/components-contrib
cd ./tests/certification/bindings/azure/blobstorage; go mod tidy -compat=1.17; cd -
/home/sky/work/code/dapr/components-contrib
cd ./tests/certification/bindings/azure/cosmosdb; go mod tidy -compat=1.17; cd -
/home/sky/work/code/dapr/components-contrib
cd ./tests/certification/bindings/azure/servicebusqueues; go mod tidy -compat=1.17; cd -
/home/sky/work/code/dapr/components-contrib
cd ./tests/certification; go mod tidy -compat=1.17; cd -
/home/sky/work/code/dapr/components-contrib
cd ./tests/certification/pubsub/rabbitmq; go mod tidy -compat=1.17; cd -
/home/sky/work/code/dapr/components-contrib
cd ./tests/certification/pubsub/mqtt; go mod tidy -compat=1.17; cd -
/home/sky/work/code/dapr/components-contrib
cd ./tests/certification/pubsub/kafka; go mod tidy -compat=1.17; cd -
/home/sky/work/code/dapr/components-contrib
cd ./tests/certification/secretstores/azure/keyvault; go mod tidy -compat=1.17; cd -
/home/sky/work/code/dapr/components-contrib
cd ./tests/certification/state/sqlserver; go mod tidy -compat=1.17; cd -
/home/sky/work/code/dapr/components-contrib
Target: check-diff
################################################################################
# Target: check-diff #
################################################################################
.PHONY: check-diff
check-diff:
.PHONY: check-diff
check-diff:
git diff --exit-code -- '*go.mod' # check no changes
git diff --exit-code -- '*go.sum' # check no changes
和 kit 仓库的 check-diff 仅仅检查 go.mod 文件不同，component-contrib 的 check-diff 还会检查 go.sum。
疑问： 为什么 kit 仓库的 check-diff 不检查 go.sum 文件？
Target: conf-tests
执行一致性测试, 其中一致性的测试案例被标注有:
//go:build conftests
// +build conftests
总共有四个文件，都在 ./tests/conformance 目录下：
- binding_test.go
- pubsub_test.go
- secretstores_test.go
- state_test.go
################################################################################
# Target: conf-tests #
################################################################################
.PHONY: conf-tests
conf-tests:
CGO_ENABLED=$(CGO) go test -v -tags=conftests -count=1 ./tests/conformance
但这些测试在本地是跑不起来的，因为缺乏对应的可连接的外部组件如 azure.eventhubs 、 azure.cosmosdb。
Target: e2e-tests-zeebe
端到端测试，类似：
################################################################################
# Target: e2e-tests-zeebe #
################################################################################
.PHONY: e2e-tests-zeebe
e2e-tests-zeebe:
CGO_ENABLED=$(CGO) go test -v -tags=e2etests -count=1 ./tests/e2e/bindings/zeebe/...
2.3 - codedov文件
codedov的配置文件
components-contrib仓库配置
https://github.com/dapr/components-contrib/blob/master/.codecov.yaml
kit仓库的 .codecov.yaml 文件极其简单，只配置了 coverage.status 的少数属性：
coverage:
# Commit status https://docs.codecov.io/docs/commit-status are used
# to block PR based on coverage threshold.
status:
project:
default:
informational: true
patch:
# Disable the coverage threshold of the patch, so that PRs are
# only failing because of overall project coverage threshold.
# See https://docs.codecov.io/docs/commit-status#disabling-a-status.
default: false
comment:
# Delete old comment and post new one for new coverage information.
behavior: new
和 kit 仓库的很不一样。
TODO： 后面来看为啥差别这么大。
2.4 - components-contrib仓库的workflow
Dapr构建学习之components-contrib仓库的workflow
2.4.1 - components-contrib.yml
components-contrib workflow
components-contrib 仓库的 workflow 有多个：
https://github.com/dapr/components-contrib/tree/master/.github/workflows
先看最核心的 components-contrib.yml
https://github.com/dapr/components-contrib/blob/master/.github/workflows/components-contrib.yml
action基本设置
和 kit 仓库完全一致，这块感觉应该各个go仓库都差不多。
steps配置
step: Set up job
这是 github action 自身的job，主要是准备操作系统和相关的软件, 以及准备相关的 actions，以备后续步骤使用：
Prepare workflow directory
Prepare all required actions
Getting action download info
Download action repository 'fkirc/skip-duplicate-actions@v3.4.0' (SHA:4c656bbdb6906310fa6213604828008bc28fe55d)
Download action repository 'actions/setup-go@v2' (SHA:424fc82d43fa5a37540bae62709ddcc23d9520d4)
Download action repository 'actions/checkout@v2' (SHA:ec3a7ce113134d7a93b817d10a8272cb61118579)
Download action repository 'golangci/golangci-lint-action@v2.2.1' (SHA:54a84d46fb7183443c869b1b7d0dc34f640fcfd7)
Download action repository 'codecov/codecov-action@v1' (SHA:29386c70ef20e286228c72b668a06fd0e8399192)
和 kit 相比，fkirc/skip-duplicate-actions@v3.4.0 是个新的内容，其他都相同。
step: Check if need skip
和 kit 仓库相比，这是最大的不同，多了一个是否需要跳过的检查，在之后的多个步骤中，都相应的增加了对是否可以跳过的判断。
- name: Check if need skip
id: skip_check
uses: fkirc/skip-duplicate-actions@v3.4.0
with:
cancel_others: 'true'
paths_ignore: '["**.md", ".codecov.yaml", ".github/workflows/dapr-automerge.yml"]'
好处就是可以极大的提高CI执行的速度，因为如果判断出来可以跳过（之前执行过检查而之后没有文件变化），就会让后续的各个步骤直接跳过执行过程从而让CI极快的通过。
这是正常执行的 workflow 的情况, 绿色框中的步骤都正常执行，总耗时3分钟37秒：
这是跳过之后的 workflow 的执行情况，可以看到绿色框中的步骤都被跳过，然后总耗时就从之前的3分钟37秒降到4秒：
检查一下日志，看这个step是怎么执行的：
Run fkirc/skip-duplicate-actions@v3.4.0
with:
cancel_others: true
paths_ignore: ["**.md", ".codecov.yaml", ".github/workflows/dapr-automerge.yml"]
github_token: ***
paths: []
skip_after_successful_duplicate: true # 在成功之后的重复就跳过
do_not_skip: ["workflow_dispatch", "schedule"]
concurrent_skipping: never
env:
GOVER: 1.17
GOOS: linux
GOARCH: amd64
GOPROXY: https://proxy.golang.org
GOLANGCI_LINT_VER: v1.31
Did not find other workflow-runs to be cancelled
Skip execution because the exact same files have been successfully checked in https://github.com/dapr/components-contrib/actions/runs/1715733057 # 发现之前有成功的检查，而文件和当前完全相同
学习一下 fkirc/skip-duplicate-actions ：
https://github.com/fkirc/skip-duplicate-actions
skip-duplicate-actions 提供了以下功能来优化GitHub Actions:
- 在合并、拉动请求或类似情况下，跳过重复的工作流运行。
- 跳过并发的或平行的工作流运行，以避免运行两次。
- 跳过忽略的路径，以加快文档更改或类似的速度。
- 如果路径没有改变，则跳过目录特定的测试等内容。
- 在分支推送后取消过时的工作流运行。
所有这些功能都有助于节省时间和成本；特别是对于长期运行的工作流程。你可以选择这些功能的任何子集。
step: Set up Go
安装go的步骤，使用到前面准备的 action actions/setup-go@v1:
- name: Set up Go ${{ env.GOVER }}
if: ${{ steps.skip_check.outputs.should_skip != 'true' }}
uses: actions/setup-go@v2
with:
go-version: ${{ env.GOVER }}
和 kit 仓库相比，只是多了一个 steps.skip_check.outputs.should_skip != 'true' 的判断，其他是一样的。
step: Check out code
checkout 代码的步骤，使用到前面准备的 action actions/checkout@v2:
- name: Check out code into the Go module directory
if: ${{ steps.skip_check.outputs.should_skip != 'true' }}
uses: actions/checkout@v2
和 kit 仓库相比，只是多了一个 steps.skip_check.outputs.should_skip != 'true' 的判断，其他是一样的。
step：Run golangci-lint
执行golangci-lint的步骤，使用到前面准备的 action golangci/golangci-lint-action@v2.2.1:
- name: Run golangci-lint
if: matrix.target_arch == 'amd64' && matrix.target_os == 'linux' && steps.skip_check.outputs.should_skip != 'true'
uses: golangci/golangci-lint-action@v2.2.1
with:
version: ${{ env.GOLANGCI_LINT_VER }}
和 kit 仓库相比，只是多了一个 steps.skip_check.outputs.should_skip != 'true' 的判断，其他是一样的。
step: Run go mod tidy check diff
使用到 Makefile 中定义的 target modtidy-all 和 check-diff:
- name: Run go mod tidy check diff
if: matrix.target_arch == 'amd64' && matrix.target_os == 'linux' && steps.skip_check.outputs.should_skip != 'true'
run: make modtidy-all check-diff
和 kit 仓库相比, 除了需要检查的 go.mod 文件是多个之外，只是多了一个 steps.skip_check.outputs.should_skip != 'true' 的判断，其他是一样的。
step: Run make test
- name: Run make test
env:
COVERAGE_OPTS: "-coverprofile=coverage.txt -covermode=atomic"
if: matrix.target_arch != 'arm' && steps.skip_check.outputs.should_skip != 'true'
run: make test
和 kit 仓库相比，只是多了一个 steps.skip_check.outputs.should_skip != 'true' 的判断，其他是一样的。
step: Codecov
step: Post Run golangci-lint
step: Post Check out code
和 kit 仓库相同。
总结
components-contrib 的 workflow 和 kit 仓库基本相同，只是多了一个 skip-duplicate-actions 的优化。
备注： 估计是 kit 仓库还没有来得及做这个 skip-duplicate-actions 优化，如果做了，就又一致了。
3.2 - Makefile文件
定义和获取变量，定义各种 target
https://github.com/dapr/dapr/blob/master/Makefile
变量定义
和 kit/components-contrib 仓库一致内容就跳过了，只看新增的内容。
# 需要打包的二进制模块，默认是这五个
BINARIES ?= daprd placement operator injector sentry
# 默认不开启 HA 模式
HA_MODE ?= false
# Force in-memory log for placement
FORCE_INMEM ?= true
# Add latest tag if LATEST_RELEASE is true
LATEST_RELEASE ?=
PROTOC ?=protoc
# name of protoc-gen-go when protoc-gen-go --version is run.
PROTOC_GEN_GO_NAME = "protoc-gen-go"
ifdef REL_VERSION
DAPR_VERSION := $(REL_VERSION)
else
DAPR_VERSION := edge
endif
和二进制打包相关的定义：
ifeq ($(GOARCH),amd64)
LATEST_TAG=latest
else
LATEST_TAG=latest-$(GOARCH)
endif
LOCAL_OS := $(shell uname)
ifeq ($(LOCAL_OS),Linux)
TARGET_OS_LOCAL = linux
else ifeq ($(LOCAL_OS),Darwin)
TARGET_OS_LOCAL = darwin
else
TARGET_OS_LOCAL ?= windows
PROTOC_GEN_GO_NAME := "protoc-gen-go.exe"
endif
export GOOS ?= $(TARGET_OS_LOCAL)
PROTOC_GEN_GO_NAME+= "v1.26.0"
# Default docker container and e2e test targst.
TARGET_OS ?= linux
TARGET_ARCH ?= amd64
TEST_OUTPUT_FILE_PREFIX ?= ./test_report
ifeq ($(GOOS),windows)
BINARY_EXT_LOCAL:=.exe
GOLANGCI_LINT:=golangci-lint.exe
export ARCHIVE_EXT = .zip
else
BINARY_EXT_LOCAL:=
GOLANGCI_LINT:=golangci-lint
export ARCHIVE_EXT = .tar.gz
endif
export BINARY_EXT ?= $(BINARY_EXT_LOCAL)
OUT_DIR := ./dist
和 helm 相关的定义：
# Helm template and install setting
HELM:=helm
RELEASE_NAME?=dapr
DAPR_NAMESPACE?=dapr-system
DAPR_MTLS_ENABLED?=true
HELM_CHART_ROOT:=./charts
HELM_CHART_DIR:=$(HELM_CHART_ROOT)/dapr
HELM_OUT_DIR:=$(OUT_DIR)/install
HELM_MANIFEST_FILE:=$(HELM_OUT_DIR)/$(RELEASE_NAME).yaml
HELM_REGISTRY?=daprio.azurecr.io
和 go build 相关的定义：
################################################################################
# Go build details #
################################################################################
BASE_PACKAGE_NAME := github.com/dapr/dapr
LOGGER_PACKAGE_NAME := github.com/dapr/kit/logger
DEFAULT_LDFLAGS:=-X $(BASE_PACKAGE_NAME)/pkg/version.gitcommit=$(GIT_COMMIT) \
-X $(BASE_PACKAGE_NAME)/pkg/version.gitversion=$(GIT_VERSION) \
-X $(BASE_PACKAGE_NAME)/pkg/version.version=$(DAPR_VERSION) \
-X $(LOGGER_PACKAGE_NAME).DaprVersion=$(DAPR_VERSION)
ifeq ($(origin DEBUG), undefined)
BUILDTYPE_DIR:=release
LDFLAGS:="$(DEFAULT_LDFLAGS) -s -w"
else ifeq ($(DEBUG),0)
BUILDTYPE_DIR:=release
LDFLAGS:="$(DEFAULT_LDFLAGS) -s -w"
else
BUILDTYPE_DIR:=debug
GCFLAGS:=-gcflags="all=-N -l"
LDFLAGS:="$(DEFAULT_LDFLAGS)"
$(info Build with debugger information)
endif
DAPR_OUT_DIR := $(OUT_DIR)/$(GOOS)_$(GOARCH)/$(BUILDTYPE_DIR)
DAPR_LINUX_OUT_DIR := $(OUT_DIR)/linux_$(GOARCH)/$(BUILDTYPE_DIR)
build target
然后就是正式的 build target 定义了。
其中 target lint / target check-diff 和 components-contrib 仓库完全一致，跳过。
Target: build
# 前面的 BINARIES 定义
BINARIES ?= daprd placement operator injector sentry
BUILDTYPE_DIR:=release 或着 debug
export BINARY_EXT ?= $(BINARY_EXT_LOCAL) # 在windows 上构建时 BINARY_EXT_LOCAL 才会设置为 .exe
################################################################################
# Target: build #
################################################################################
.PHONY: build
DAPR_BINS:=$(foreach ITEM,$(BINARIES),$(DAPR_OUT_DIR)/$(ITEM)$(BINARY_EXT))
build: $(DAPR_BINS)
# Generate builds for dapr binaries for the target
# Params:
# $(1): the binary name for the target
# $(2): the binary main directory
# $(3): the target os
# $(4): the target arch
# $(5): the output directory
define genBinariesForTarget
.PHONY: $(5)/$(1)
$(5)/$(1):
CGO_ENABLED=$(CGO) GOOS=$(3) GOARCH=$(4) go build $(GCFLAGS) -ldflags=$(LDFLAGS) \
-o $(5)/$(1) $(2)/;
endef
# Generate binary targets
$(foreach ITEM,$(BINARIES),$(eval $(call genBinariesForTarget,$(ITEM)$(BINARY_EXT),./cmd/$(ITEM),$(GOOS),$(GOARCH),$(DAPR_OUT_DIR))))
循环构建 BINARIES 中定义的二进制模块。
在 macos + m1 芯片的 MacBook 上，输出如下：
make build
CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/darwin_arm64/release/daprd ./cmd/daprd/;
go: downloading github.com/dapr/components-contrib v1.6.0-rc.1.0.20220307041340-f1209fb068c7
CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/darwin_arm64/release/placement ./cmd/placement/;
CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/darwin_arm64/release/operator ./cmd/operator/;
CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/darwin_arm64/release/injector ./cmd/injector/;
CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/darwin_arm64/release/sentry ./cmd/sentry/;
构建成功的二进制文件存在放 ./dist/darwin_arm64/release/ 目录下:
Target: build-linux
################################################################################
# Target: build-linux #
################################################################################
BUILD_LINUX_BINS:=$(foreach ITEM,$(BINARIES),$(DAPR_LINUX_OUT_DIR)/$(ITEM))
build-linux: $(BUILD_LINUX_BINS)
# Generate linux binaries targets to build linux docker image
ifneq ($(GOOS), linux)
$(foreach ITEM,$(BINARIES),$(eval $(call genBinariesForTarget,$(ITEM),./cmd/$(ITEM),linux,$(GOARCH),$(DAPR_LINUX_OUT_DIR))))
endif
在 macos + m1 芯片的 MacBook 上，输出如下：
$ make build-linux
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/linux_arm64/release/daprd ./cmd/daprd/;
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/linux_arm64/release/placement ./cmd/placement/;
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/linux_arm64/release/operator ./cmd/operator/;
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/linux_arm64/release/injector ./cmd/injector/;
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/linux_arm64/release/sentry ./cmd/sentry/;
构建成功的二进制文件存在放 ./dist/linux_arm64/release/ 目录下:
注意是 arm64，不是 amd64.
Target: archive
# 相关的变量定义
OUT_DIR := ./dist
DAPR_OUT_DIR := $(OUT_DIR)/$(GOOS)_$(GOARCH)/$(BUILDTYPE_DIR)
DAPR_LINUX_OUT_DIR := $(OUT_DIR)/linux_$(GOARCH)/$(BUILDTYPE_DIR)
################################################################################
# Target: archive #
################################################################################
ARCHIVE_OUT_DIR ?= $(DAPR_OUT_DIR)
ARCHIVE_FILE_EXTS:=$(foreach ITEM,$(BINARIES),archive-$(ITEM)$(ARCHIVE_EXT))
archive: $(ARCHIVE_FILE_EXTS)
# Generate archive files for each binary
# $(1): the binary name to be archived
# $(2): the archived file output directory
define genArchiveBinary
ifeq ($(GOOS),windows)
archive-$(1).zip:
7z.exe a -tzip "$(2)\\$(1)_$(GOOS)_$(GOARCH)$(ARCHIVE_EXT)" "$(DAPR_OUT_DIR)\\$(1)$(BINARY_EXT)"
else
archive-$(1).tar.gz:
tar czf "$(2)/$(1)_$(GOOS)_$(GOARCH)$(ARCHIVE_EXT)" -C "$(DAPR_OUT_DIR)" "$(1)$(BINARY_EXT)"
endif
endef
# Generate archive-*.[zip|tar.gz] targets
$(foreach ITEM,$(BINARIES),$(eval $(call genArchiveBinary,$(ITEM),$(ARCHIVE_OUT_DIR))))
打包为 ta r.gz 或者 zip 文件。
在 macos + m1 芯片的 MacBook 上，输出如下：
$ make archive
tar czf "./dist/darwin_arm64/release/daprd_darwin_arm64.tar.gz" -C "./dist/darwin_arm64/release" "daprd"
tar czf "./dist/darwin_arm64/release/placement_darwin_arm64.tar.gz" -C "./dist/darwin_arm64/release" "placement"
tar czf "./dist/darwin_arm64/release/operator_darwin_arm64.tar.gz" -C "./dist/darwin_arm64/release" "operator"
tar czf "./dist/darwin_arm64/release/injector_darwin_arm64.tar.gz" -C "./dist/darwin_arm64/release" "injector"
tar czf "./dist/darwin_arm64/release/sentry_darwin_arm64.tar.gz" -C "./dist/darwin_arm64/release" "sentry"
Target: manifest-gen
################################################################################
# Target: manifest-gen #
################################################################################
# Generate helm chart manifest
manifest-gen: dapr.yaml
dapr.yaml: check-docker-env
$(info Generating helm manifest $(HELM_MANIFEST_FILE)...)
@mkdir -p $(HELM_OUT_DIR)
$(HELM) template \
--include-crds=true --set global.ha.enabled=$(HA_MODE) --set dapr_config.dapr_config_chart_included=false --set-string global.tag=$(DAPR_TAG) --set-string global.registry=$(DAPR_REGISTRY) $(HELM_CHART_DIR) > $(HELM_MANIFEST_FILE)
需要先设置 DAPR_REGISTRY 变量，如果没有设置则会报错：
make manifest-gen
docker/docker.mk:76: *** DAPR_REGISTRY environment variable must be set. Stop.
Target: Target: upload-helmchart
################################################################################
# Target: upload-helmchart
################################################################################
# Upload helm charts to Helm Registry
upload-helmchart:
export HELM_EXPERIMENTAL_OCI=1; \
$(HELM) chart save ${HELM_CHART_ROOT}/${RELEASE_NAME} ${HELM_REGISTRY}/${HELM}/${RELEASE_NAME}:${DAPR_VERSION}; \
$(HELM) chart push ${HELM_REGISTRY}/${HELM}/${RELEASE_NAME}:${DAPR_VERSION}
TBD
Target: docker-deploy-k8s
################################################################################
# Target: docker-deploy-k8s #
################################################################################
PULL_POLICY?=Always
docker-deploy-k8s: check-docker-env check-arch
$(info Deploying ${DAPR_REGISTRY}/${RELEASE_NAME}:${DAPR_TAG} to the current K8S context...)
$(HELM) install \
$(RELEASE_NAME) --namespace=$(DAPR_NAMESPACE) --wait --timeout 5m0s \
--set global.ha.enabled=$(HA_MODE) --set-string global.tag=$(DAPR_TAG)-$(TARGET_OS)-$(TARGET_ARCH) \
--set-string global.registry=$(DAPR_REGISTRY) --set global.logAsJson=true \
--set global.daprControlPlaneOs=$(TARGET_OS) --set global.daprControlPlaneArch=$(TARGET_ARCH) \
--set dapr_placement.logLevel=debug --set dapr_sidecar_injector.sidecarImagePullPolicy=$(PULL_POLICY) \
--set global.imagePullPolicy=$(PULL_POLICY) --set global.imagePullSecrets=${DAPR_TEST_REGISTRY_SECRET} \
--set global.mtls.enabled=${DAPR_MTLS_ENABLED} \
--set dapr_placement.cluster.forceInMemoryLog=$(FORCE_INMEM) $(HELM_CHART_DIR)
TBD
Target: docker-deploy-k8s
################################################################################
# Target: docker-deploy-k8s #
################################################################################
PULL_POLICY?=Always
docker-deploy-k8s: check-docker-env check-arch
$(info Deploying ${DAPR_REGISTRY}/${RELEASE_NAME}:${DAPR_TAG} to the current K8S context...)
$(HELM) install \
$(RELEASE_NAME) --namespace=$(DAPR_NAMESPACE) --wait --timeout 5m0s \
--set global.ha.enabled=$(HA_MODE) --set-string global.tag=$(DAPR_TAG)-$(TARGET_OS)-$(TARGET_ARCH) \
--set-string global.registry=$(DAPR_REGISTRY) --set global.logAsJson=true \
--set global.daprControlPlaneOs=$(TARGET_OS) --set global.daprControlPlaneArch=$(TARGET_ARCH) \
--set dapr_placement.logLevel=debug --set dapr_sidecar_injector.sidecarImagePullPolicy=$(PULL_POLICY) \
--set global.imagePullPolicy=$(PULL_POLICY) --set global.imagePullSecrets=${DAPR_TEST_REGISTRY_SECRET} \
--set global.mtls.enabled=${DAPR_MTLS_ENABLED} \
--set dapr_placement.cluster.forceInMemoryLog=$(FORCE_INMEM) $(HELM_CHART_DIR)
TBD
Target: test
################################################################################
# Target: test #
################################################################################
.PHONY: test
test: test-deps
gotestsum --jsonfile $(TEST_OUTPUT_FILE_PREFIX)_unit.json --format standard-quiet -- ./pkg/... ./utils/... ./cmd/... $(COVERAGE_OPTS)
go test ./tests/...
TBD
在 macos + m1 芯片的 MacBook 上，输出如下：
make test
# The desire here is to download this test dependency without polluting go.mod
# In golang >=1.16 there is a new way to do this with `go install gotest.tools/gotestsum@latest`
# But this doesn't work with <=1.15.
# (see: https://golang.org/ref/mod#go-install)
command -v gotestsum || go install gotest.tools/gotestsum@latest
/Users/sky/work/soft/gopath/bin/gotestsum
gotestsum --jsonfile ./test_report_unit.json --format standard-quiet -- ./pkg/... ./utils/... ./cmd/...
ok github.com/dapr/dapr/pkg/acl 0.850s
ok github.com/dapr/dapr/pkg/apis/subscriptions/v2alpha1 (cached)
ok github.com/dapr/dapr/pkg/components/bindings 0.769s
ok github.com/dapr/dapr/pkg/components/configuration 1.025s
......
ok github.com/dapr/dapr/pkg/actors/internal 15.492s
ok github.com/dapr/dapr/pkg/health 14.228s
ok github.com/dapr/dapr/pkg/placement 10.674s
ok github.com/dapr/dapr/pkg/sentry/ca 8.499s
......
# actor的测试实在是太夸张了
ok github.com/dapr/dapr/pkg/actors 220.119s
......
DONE 1369 tests in 231.313s
Tareget: init-proto
################################################################################
# Target: init-proto #
################################################################################
.PHONY: init-proto
init-proto:
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.26
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1
初始化安装 protoc-gen-go 和 protoc-gen-go-grpc 。
$ make init-proto
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.26
go: downloading google.golang.org/protobuf v1.26.0
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1
go: downloading google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0
go: downloading google.golang.org/protobuf v1.23.0
Tareget: gen-proto
################################################################################
# Target: gen-proto #
################################################################################
# 这是各个模块的 proto 文件
GRPC_PROTOS:=common internals operator placement runtime sentry
PROTO_PREFIX:=github.com/dapr/dapr
# Generate archive files for each binary
# $(1): the binary name to be archived
define genProtoc
.PHONY: gen-proto-$(1)
gen-proto-$(1):
$(PROTOC) --go_out=. --go_opt=module=$(PROTO_PREFIX) --go-grpc_out=. --go-grpc_opt=require_unimplemented_servers=false,module=$(PROTO_PREFIX) ./dapr/proto/$(1)/v1/*.proto
endef
$(foreach ITEM,$(GRPC_PROTOS),$(eval $(call genProtoc,$(ITEM))))
GEN_PROTOS:=$(foreach ITEM,$(GRPC_PROTOS),gen-proto-$(ITEM))
.PHONY: gen-proto
gen-proto: check-proto-version $(GEN_PROTOS) modtidy
Proto 源文件存放在 dapr/dapr 仓库下的 dapr/proto 目录中，子目录名对应 GRPC_PROTOS 变量。
注意在执行前要先安装好 protoc，否则会报错：
$ make gen-proto
make: protoc: Command not found
please use protoc 3.14.0 to generate proto, see https://github.com/dapr/dapr/blob/master/dapr/README.md#proto-client-generation
make: *** [check-proto-version] Error 1
执行输出如下：
$ make gen-proto
protoc --go_out=. --go_opt=module=github.com/dapr/dapr --go-grpc_out=. --go-grpc_opt=require_unimplemented_servers=false,module=github.com/dapr/dapr ./dapr/proto/common/v1/*.proto
protoc --go_out=. --go_opt=module=github.com/dapr/dapr --go-grpc_out=. --go-grpc_opt=require_unimplemented_servers=false,module=github.com/dapr/dapr ./dapr/proto/internals/v1/*.proto
protoc --go_out=. --go_opt=module=github.com/dapr/dapr --go-grpc_out=. --go-grpc_opt=require_unimplemented_servers=false,module=github.com/dapr/dapr ./dapr/proto/operator/v1/*.proto
protoc --go_out=. --go_opt=module=github.com/dapr/dapr --go-grpc_out=. --go-grpc_opt=require_unimplemented_servers=false,module=github.com/dapr/dapr ./dapr/proto/placement/v1/*.proto
protoc --go_out=. --go_opt=module=github.com/dapr/dapr --go-grpc_out=. --go-grpc_opt=require_unimplemented_servers=false,module=github.com/dapr/dapr ./dapr/proto/runtime/v1/*.proto
protoc --go_out=. --go_opt=module=github.com/dapr/dapr --go-grpc_out=. --go-grpc_opt=require_unimplemented_servers=false,module=github.com/dapr/dapr ./dapr/proto/sentry/v1/*.proto
go mod tidy
生成的 go 文件会存放在 pkg/proto 目录下，子目录结构同样对应 GRPC_PROTOS 定义：
Target: check-proto-version
################################################################################
# Target: check-proto-version #
################################################################################
.PHONY: check-proto-version
check-proto-version: ## Checking the version of proto related tools
@test "$(shell protoc --version)" = "libprotoc 3.14.0" \
|| { echo "please use protoc 3.14.0 to generate proto, see https://github.com/dapr/dapr/blob/master/dapr/README.md#proto-client-generation"; exit 1; }
@test "$(shell protoc-gen-go-grpc --version)" = "protoc-gen-go-grpc 1.1.0" \
|| { echo "please use protoc-gen-go-grpc 1.1.0 to generate proto, see https://github.com/dapr/dapr/blob/master/dapr/README.md#proto-client-generation"; exit 1; }
@test "$(shell protoc-gen-go --version 2>&1)" = "$(PROTOC_GEN_GO_NAME)" \
|| { echo "please use protoc-gen-go v1.26.0 to generate proto, see https://github.com/dapr/dapr/blob/master/dapr/README.md#proto-client-generation"; exit 1; }
检查 proto 代码生成相关的工具的版本，因为不同版本生成的代码会有差异，如果混合不同版本使用，则这些生成的代码的 git 记录会混乱无比。
备注：这个 target 是我加的。
Target: check-proto-diff
################################################################################
# Target: check-proto-diff #
################################################################################
.PHONY: check-proto-diff
check-proto-diff:
git diff --exit-code ./pkg/proto/common/v1/common.pb.go # check no changes
git diff --exit-code ./pkg/proto/internals/v1/status.pb.go # check no changes
git diff --exit-code ./pkg/proto/operator/v1/operator.pb.go # check no changes
git diff --exit-code ./pkg/proto/operator/v1/operator_grpc.pb.go # check no changes
git diff --exit-code ./pkg/proto/runtime/v1/appcallback.pb.go # check no changes
git diff --exit-code ./pkg/proto/runtime/v1/appcallback_grpc.pb.go # check no changes
git diff --exit-code ./pkg/proto/runtime/v1/dapr.pb.go # check no changes
git diff --exit-code ./pkg/proto/runtime/v1/dapr_grpc.pb.go # check no changes
git diff --exit-code ./pkg/proto/sentry/v1/sentry.pb.go # check no changes
检查 proto 生成的代码是否有变化。
$ make check-proto-diff
git diff --exit-code ./pkg/proto/common/v1/common.pb.go # check no changes
git diff --exit-code ./pkg/proto/internals/v1/status.pb.go # check no changes
git diff --exit-code ./pkg/proto/operator/v1/operator.pb.go # check no changes
git diff --exit-code ./pkg/proto/operator/v1/operator_grpc.pb.go # check no changes
git diff --exit-code ./pkg/proto/runtime/v1/appcallback.pb.go # check no changes
git diff --exit-code ./pkg/proto/runtime/v1/appcallback_grpc.pb.go # check no changes
git diff --exit-code ./pkg/proto/runtime/v1/dapr.pb.go # check no changes
git diff --exit-code ./pkg/proto/runtime/v1/dapr_grpc.pb.go # check no changes
git diff --exit-code ./pkg/proto/sentry/v1/sentry.pb.go # check no changes
Target: get-components-contrib
################################################################################
# Target: get-components-contrib #
################################################################################
.PHONY: get-components-contrib
get-components-contrib:
go get github.com/dapr/components-contrib@master
获取 components-contrib 仓库 master 分支的最新代码。
执行结果输出如下：
$ make get-components-contrib
go get github.com/dapr/components-contrib@master
go: downloading github.com/dapr/components-contrib v1.6.0-rc.1.0.20220310012151-027204f2d3c5
go get: upgraded github.com/dapr/components-contrib v1.6.0-rc.1.0.20220307041340-f1209fb068c7 => v1.6.0-rc.1.0.20220310012151-027204f2d3c5
3.3 - Makefile codegen部分
和代码生成相关的make target
https://github.com/dapr/dapr/blob/master/Makefile
################################################################################
# Target: codegen #
################################################################################
include tools/codegen.mk
https://github.com/dapr/dapr/blob/master/tools/codegen.mk
变量定义
# 依然是这六个 proto 模块
PROTOS = operator placement sentry common runtime internals
ifeq (,$(shell go env GOBIN))
GOBIN=$(shell go env GOPATH)/bin
else
GOBIN=$(shell go env GOBIN)
endif
Target: code-generate
# Generate code
code-generate: controller-gen
$(CONTROLLER_GEN) object:headerFile="./tools/boilerplate.go.txt" \
crd:trivialVersions=true crd:crdVersions=v1 paths="./pkg/apis/..." output:crd:artifacts:config=config/crd/bases
执行结果输出如下：
$ make code-generate
go: downloading sigs.k8s.io/controller-tools v0.5.0
go: downloading github.com/spf13/cobra v1.1.1
go: downloading github.com/fatih/color v1.9.0
go: downloading golang.org/x/tools v0.0.0-20200616133436-c1934b75d054
go: downloading sigs.k8s.io/yaml v1.2.0
go: downloading k8s.io/api v0.20.2
go: downloading k8s.io/apimachinery v0.20.2
go: downloading gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776
go: downloading k8s.io/apiextensions-apiserver v0.20.2
go: downloading github.com/gobuffalo/flect v0.2.2
go: downloading gopkg.in/yaml.v2 v2.3.0
go: downloading github.com/gogo/protobuf v1.3.1
go: downloading k8s.io/utils v0.0.0-20201110183641-67b214c5f920
go: downloading k8s.io/klog/v2 v2.4.0
go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.0.2
go: downloading golang.org/x/sys v0.0.0-20201112073958-5cba982894dd
go: downloading github.com/go-logr/logr v0.2.0
go: downloading github.com/json-iterator/go v1.1.10
go: downloading golang.org/x/net v0.0.0-20201110031124-69a78807bb2b
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading golang.org/x/text v0.3.4
go get: installing executables with 'go get' in module mode is deprecated.
Use 'go install pkg@version' instead.
For more information, see https://golang.org/doc/go-get-install-deprecation
or run 'go help get' or 'go help install'.
/Users/sky/work/soft/gopath/bin/controller-gen object:headerFile="./tools/boilerplate.go.txt" \
crd:trivialVersions=true crd:crdVersions=v1 paths="./pkg/apis/..." output:crd:artifacts:config=config/crd/bases
再次运行，在 controller-gen 已经下载好的情况下日志就干净很多了：
$ make code-generate
/Users/sky/work/soft/gopath/bin/controller-gen object:headerFile="./tools/boilerplate.go.txt" \
crd:trivialVersions=true crd:crdVersions=v1 paths="./pkg/apis/..." output:crd:artifacts:config=config/crd/bases
生成的代码存放在 dapr 项目根目录下的 config 子目录：
TBD：有空再细看。
Target: controller-gen
controller-gen target的定义有三部分。首先是找到 controller-gen，如果没有就下载：
# find or download controller-gen
# download controller-gen if necessary
controller-gen:
ifeq (, $(shell which controller-gen))
@{ \
set -e ;\
CONTROLLER_GEN_TMP_DIR="$$(mktemp -d)" ;\
cd "$$CONTROLLER_GEN_TMP_DIR" ;\
GO111MODULE=on go get sigs.k8s.io/controller-tools/cmd/controller-gen@v0.5.0 ; \
rm -rf "$$CONTROLLER_GEN_TMP_DIR" ;\
}
CONTROLLER_GEN=$(GOBIN)/controller-gen
else
CONTROLLER_GEN=$(shell which controller-gen)
endif
然后是一段方法定义, 动态定义了多个 protoc-gen-$(1)-v1 target：
define genProtoForTarget
.PHONY: $(1)
protoc-gen-$(1)-v1:
protoc -I . ./dapr/proto/$(1)/v1/*.proto --go_out=plugins=grpc:.
cp -R ./github.com/dapr/dapr/pkg/proto/$(1)/v1/*.go ./pkg/proto/$(1)/v1
rm -rf ./github.com
endef
循环 PROTOS，为每个模块执行 protoc-gen-$(1)-v1 target：
PROTOS = operator placement sentry common runtime internals
# Generate proto gen targets
$(foreach ITEM,$(PROTOS),$(eval $(call genProtoForTarget,$(ITEM))))
PROTOC_ALL_TARGETS:=$(foreach ITEM,$(PROTOS),protoc-gen-$(ITEM)-v1)
protoc-gen: $(PROTOC_ALL_TARGETS)
3.4 - Makefile docker部分
和 docker 相关的 make target
https://github.com/dapr/dapr/blob/master/Makefile
################################################################################
# Target: docker #
################################################################################
include docker/docker.mk
https://github.com/dapr/dapr/blob/master/docker/docker.mk
变量定义
image相关的变量定义：
# Docker image build and push setting
DOCKER:=docker
DOCKERFILE_DIR?=./docker
DAPR_SYSTEM_IMAGE_NAME=$(RELEASE_NAME)
DAPR_RUNTIME_IMAGE_NAME=daprd
DAPR_PLACEMENT_IMAGE_NAME=placement
DAPR_SENTRY_IMAGE_NAME=sentry
# build docker image for linux
BIN_PATH=$(OUT_DIR)/$(TARGET_OS)_$(TARGET_ARCH)
ifeq ($(TARGET_OS), windows)
DOCKERFILE:=Dockerfile-windows
BIN_PATH := $(BIN_PATH)/release
else ifeq ($(origin DEBUG), undefined)
DOCKERFILE:=Dockerfile
BIN_PATH := $(BIN_PATH)/release
else ifeq ($(DEBUG),0)
DOCKERFILE:=Dockerfile
BIN_PATH := $(BIN_PATH)/release
else
DOCKERFILE:=Dockerfile-debug
BIN_PATH := $(BIN_PATH)/debug
endif
ifeq ($(TARGET_ARCH),arm)
DOCKER_IMAGE_PLATFORM:=$(TARGET_OS)/arm/v7
else ifeq ($(TARGET_ARCH),arm64)
DOCKER_IMAGE_PLATFORM:=$(TARGET_OS)/arm64/v8
else
DOCKER_IMAGE_PLATFORM:=$(TARGET_OS)/amd64
endif
支持的 cpu 架构：
# Supported docker image architecture
DOCKERMUTI_ARCH=linux-amd64 linux-arm linux-arm64 windows-amd64
还有为 target docker-build 和 docker-push 定义的变量：
LINUX_BINS_OUT_DIR=$(OUT_DIR)/linux_$(GOARCH)
DOCKER_IMAGE_TAG=$(DAPR_REGISTRY)/$(DAPR_SYSTEM_IMAGE_NAME):$(DAPR_TAG)
DAPR_RUNTIME_DOCKER_IMAGE_TAG=$(DAPR_REGISTRY)/$(DAPR_RUNTIME_IMAGE_NAME):$(DAPR_TAG)
DAPR_PLACEMENT_DOCKER_IMAGE_TAG=$(DAPR_REGISTRY)/$(DAPR_PLACEMENT_IMAGE_NAME):$(DAPR_TAG)
DAPR_SENTRY_DOCKER_IMAGE_TAG=$(DAPR_REGISTRY)/$(DAPR_SENTRY_IMAGE_NAME):$(DAPR_TAG)
ifeq ($(LATEST_RELEASE),true)
DOCKER_IMAGE_LATEST_TAG=$(DAPR_REGISTRY)/$(DAPR_SYSTEM_IMAGE_NAME):$(LATEST_TAG)
DAPR_RUNTIME_DOCKER_IMAGE_LATEST_TAG=$(DAPR_REGISTRY)/$(DAPR_RUNTIME_IMAGE_NAME):$(LATEST_TAG)
DAPR_PLACEMENT_DOCKER_IMAGE_LATEST_TAG=$(DAPR_REGISTRY)/$(DAPR_PLACEMENT_IMAGE_NAME):$(LATEST_TAG)
DAPR_SENTRY_DOCKER_IMAGE_LATEST_TAG=$(DAPR_REGISTRY)/$(DAPR_SENTRY_IMAGE_NAME):$(LATEST_TAG)
endif
# To use buildx: https://github.com/docker/buildx#docker-ce
export DOCKER_CLI_EXPERIMENTAL=enabled
Target: check-docker-env
# check the required environment variables
check-docker-env:
ifeq ($(DAPR_REGISTRY),)
$(error DAPR_REGISTRY environment variable must be set)
endif
ifeq ($(DAPR_TAG),)
$(error DAPR_TAG environment variable must be set)
endif
检查 DAPR_REGISTRY 和 DAPR_TAG 两个环境变量是否设置，验证如下：
$ make check-docker-env
docker/docker.mk:76: *** DAPR_REGISTRY environment variable must be set. Stop.
$ export DAPR_REGISTRY=docker.io/skyao
$ make check-docker-env
docker/docker.mk:79: *** DAPR_TAG environment variable must be set. Stop.
$ export DAPR_TAG=dev
$ make check-docker-env
make: Nothing to be done for `check-docker-env'.
Target: check-docker-env
check-arch:
ifeq ($(TARGET_OS),)
$(error TARGET_OS environment variable must be set)
endif
ifeq ($(TARGET_ARCH),)
$(error TARGET_ARCH environment variable must be set)
endif
检查 TARGET_OS 和 TARGET_ARCH 两个环境变量是否设置，验证如下：
$ make check-arch
make: Nothing to be done for `check-arch'.
Target: docker-build
在执行 docker-build 之前，最好先设置好相关的环境变量：
export DAPR_REGISTRY=docker.io/skyao
export DAPR_TAG=dev # 这个先别设置，用默认值
export TARGET_OS=linux # 默认是linux
export TARGET_ARCH=arm64 # 默认是amd64，m1上需要修改
然后执行：
docker-build: check-docker-env check-arch
$(info Building $(DOCKER_IMAGE_TAG) docker image ...)
ifeq ($(TARGET_ARCH),amd64)
$(DOCKER) build --build-arg PKG_FILES=* -f $(DOCKERFILE_DIR)/$(DOCKERFILE) $(BIN_PATH) -t $(DOCKER_IMAGE_TAG)-$(TARGET_OS)-$(TARGET_ARCH)
$(DOCKER) build --build-arg PKG_FILES=daprd -f $(DOCKERFILE_DIR)/$(DOCKERFILE) $(BIN_PATH) -t $(DAPR_RUNTIME_DOCKER_IMAGE_TAG)-$(TARGET_OS)-$(TARGET_ARCH)
$(DOCKER) build --build-arg PKG_FILES=placement -f $(DOCKERFILE_DIR)/$(DOCKERFILE) $(BIN_PATH) -t $(DAPR_PLACEMENT_DOCKER_IMAGE_TAG)-$(TARGET_OS)-$(TARGET_ARCH)
$(DOCKER) build --build-arg PKG_FILES=sentry -f $(DOCKERFILE_DIR)/$(DOCKERFILE) $(BIN_PATH) -t $(DAPR_SENTRY_DOCKER_IMAGE_TAG)-$(TARGET_OS)-$(TARGET_ARCH)
else
-$(DOCKER) buildx create --use --name daprbuild
-$(DOCKER) run --rm --privileged multiarch/qemu-user-static --reset -p yes
$(DOCKER) buildx build --build-arg PKG_FILES=* --platform $(DOCKER_IMAGE_PLATFORM) -f $(DOCKERFILE_DIR)/$(DOCKERFILE) $(BIN_PATH) -t $(DOCKER_IMAGE_TAG)-$(TARGET_OS)-$(TARGET_ARCH)
$(DOCKER) buildx build --build-arg PKG_FILES=daprd --platform $(DOCKER_IMAGE_PLATFORM) -f $(DOCKERFILE_DIR)/$(DOCKERFILE) $(BIN_PATH) -t $(DAPR_RUNTIME_DOCKER_IMAGE_TAG)-$(TARGET_OS)-$(TARGET_ARCH)
$(DOCKER) buildx build --build-arg PKG_FILES=placement --platform $(DOCKER_IMAGE_PLATFORM) -f $(DOCKERFILE_DIR)/$(DOCKERFILE) $(BIN_PATH) -t $(DAPR_PLACEMENT_DOCKER_IMAGE_TAG)-$(TARGET_OS)-$(TARGET_ARCH)
$(DOCKER) buildx build --build-arg PKG_FILES=sentry --platform $(DOCKER_IMAGE_PLATFORM) -f $(DOCKERFILE_DIR)/$(DOCKERFILE) $(BIN_PATH) -t $(DAPR_SENTRY_DOCKER_IMAGE_TAG)-$(TARGET_OS)-$(TARGET_ARCH)
endif
在执行 docker-build 之前，需要先执行 target build-linux，不然会提示：
$ make docker-build
Building docker.io/skyao/dapr:dev docker image ...
docker build --build-arg PKG_FILES=* -f ./docker/Dockerfile ./dist/linux_amd64/release -t docker.io/skyao/dapr:dev-linux-amd64
unable to prepare context: path "./dist/linux_amd64/release" not found
make: *** [docker-build] Error 1
m1上的构建
target build-linux 会构建 linux_arm64 版本的二进制文件：
$ make build-linux
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/linux_arm64/release/daprd ./cmd/daprd/;
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/linux_arm64/release/placement ./cmd/placement/;
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/linux_arm64/release/operator ./cmd/operator/;
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/linux_arm64/release/injector ./cmd/injector/;
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-X github.com/dapr/dapr/pkg/version.gitcommit=551722f533afa5dfee97482fe3e63d8ff6233d50 -X github.com/dapr/dapr/pkg/version.gitversion=v1.5.1-rc.3-411-g551722f -X github.com/dapr/dapr/pkg/version.version=edge -X github.com/dapr/kit/logger.DaprVersion=edge -s -w" -o ./dist/linux_arm64/release/sentry ./cmd/sentry/;
在 m1 上执行 docker-build ：
$ make docker-build
Building docker.io/skyao/dapr:dev docker image ...
docker buildx create --use --name daprbuild
error: existing instance for daprbuild but no append mode, specify --node to make changes for existing instances
make: [docker-build] Error 1 (ignored)
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
Unable to find image 'multiarch/qemu-user-static:latest' locally
latest: Pulling from multiarch/qemu-user-static
01c2cdc13739: Pull complete
11933eee4160: Pull complete
30abb83a18eb: Pull complete
0657daef200b: Pull complete
10094524a9f3: Pull complete
Digest: sha256:2c8b8fcf1d6badfca797c3fb46b7bb5f705ec7e66363e1cfeb7b7d4c7086e360
Status: Downloaded newer image for multiarch/qemu-user-static:latest
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Error while loading /qemu-binfmt-conf.sh: Exec format error
make: [docker-build] Error 1 (ignored)
docker buildx build --build-arg PKG_FILES=* --platform linux/arm64/v8 -f ./docker/Dockerfile ./dist/linux_arm64/release -t docker.io/skyao/dapr:dev-linux-arm64
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 84.8s (12/12) FINISHED
=> [internal] booting buildkit 69.4s
=> => pulling image moby/buildkit:buildx-stable-1 69.0s
=> => creating container buildx_buildkit_daprbuild0 0.4s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 297B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for gcr.io/distroless/static:nonroot 4.7s
=> [internal] load metadata for docker.io/library/alpine:latest 6.9s
=> [auth] library/alpine:pull token for registry-1.docker.io 0.0s
=> [alpine 1/2] FROM docker.io/library/alpine:latest@sha256:ceeae2849a425ef1a7e591d8288f1a58cdf1f4e8d9da7510e29ea829e61cf512 0.2s
=> => resolve docker.io/library/alpine:latest@sha256:ceeae2849a425ef1a7e591d8288f1a58cdf1f4e8d9da7510e29ea829e61cf512 0.0s
=> => sha256:a5e44472bb1f0d721d23f82fa10a4c3d25994790238a173c1de950a649eb9a90 2.71MB / 2.71MB 2.9s
=> => extracting sha256:a5e44472bb1f0d721d23f82fa10a4c3d25994790238a173c1de950a649eb9a90 0.2s
=> [internal] load build context 7.4s
=> => transferring context: 233.03MB 7.3s
=> [stage-1 1/4] FROM gcr.io/distroless/static:nonroot@sha256:80c956fb0836a17a565c43a4026c9c80b2013c83bea09f74fa4da195a59b7a99 0.2s
=> => resolve gcr.io/distroless/static:nonroot@sha256:80c956fb0836a17a565c43a4026c9c80b2013c83bea09f74fa4da195a59b7a99 0.0s
=> => sha256:dbcab61d5a5a806aee6156f2e22c601a52119ca8eaeb8fcd08187f22c35d9b88 803.83kB / 803.83kB 3.6s
=> => extracting sha256:dbcab61d5a5a806aee6156f2e22c601a52119ca8eaeb8fcd08187f22c35d9b88 0.2s
=> [alpine 2/2] RUN apk add -U --no-cache ca-certificates 4.5s
=> [stage-1 2/4] COPY --from=alpine /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ 0.0s
=> [stage-1 3/4] COPY /* / 0.2s
docker buildx build --build-arg PKG_FILES=daprd --platform linux/arm64/v8 -f ./docker/Dockerfile ./dist/linux_arm64/release -t docker.io/skyao/daprd:dev-linux-arm64
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 1.3s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 297B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for gcr.io/distroless/static:nonroot 0.9s
=> [internal] load metadata for docker.io/library/alpine:latest 0.5s
=> [alpine 1/2] FROM docker.io/library/alpine:latest@sha256:ceeae2849a425ef1a7e591d8288f1a58cdf1f4e8d9da7510e29ea829e61cf512 0.0s
=> => resolve docker.io/library/alpine:latest@sha256:ceeae2849a425ef1a7e591d8288f1a58cdf1f4e8d9da7510e29ea829e61cf512 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 29B 0.0s
=> [stage-1 1/4] FROM gcr.io/distroless/static:nonroot@sha256:80c956fb0836a17a565c43a4026c9c80b2013c83bea09f74fa4da195a59b7a99 0.0s
=> => resolve gcr.io/distroless/static:nonroot@sha256:80c956fb0836a17a565c43a4026c9c80b2013c83bea09f74fa4da195a59b7a99 0.0s
=> CACHED [alpine 2/2] RUN apk add -U --no-cache ca-certificates 0.0s
=> CACHED [stage-1 2/4] COPY --from=alpine /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ 0.0s
=> [stage-1 3/4] COPY /daprd / 0.1s
docker buildx build --build-arg PKG_FILES=placement --platform linux/arm64/v8 -f ./docker/Dockerfile ./dist/linux_arm64/release -t docker.io/skyao/placement:dev-linux-arm64
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 1.9s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 297B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for gcr.io/distroless/static:nonroot 1.2s
=> [internal] load metadata for docker.io/library/alpine:latest 0.5s
=> [alpine 1/2] FROM docker.io/library/alpine:latest@sha256:ceeae2849a425ef1a7e591d8288f1a58cdf1f4e8d9da7510e29ea829e61cf512 0.0s
=> => resolve docker.io/library/alpine:latest@sha256:ceeae2849a425ef1a7e591d8288f1a58cdf1f4e8d9da7510e29ea829e61cf512 0.0s
=> [internal] load build context 0.5s
=> => transferring context: 14.42MB 0.5s
=> [stage-1 1/4] FROM gcr.io/distroless/static:nonroot@sha256:80c956fb0836a17a565c43a4026c9c80b2013c83bea09f74fa4da195a59b7a99 0.0s
=> => resolve gcr.io/distroless/static:nonroot@sha256:80c956fb0836a17a565c43a4026c9c80b2013c83bea09f74fa4da195a59b7a99 0.0s
=> CACHED [alpine 2/2] RUN apk add -U --no-cache ca-certificates 0.0s
=> CACHED [stage-1 2/4] COPY --from=alpine /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ 0.0s
=> [stage-1 3/4] COPY /placement / 0.0s
docker buildx build --build-arg PKG_FILES=sentry --platform linux/arm64/v8 -f ./docker/Dockerfile ./dist/linux_arm64/release -t docker.io/skyao/sentry:dev-linux-arm64
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 2.5s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 297B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for gcr.io/distroless/static:nonroot 1.0s
=> [internal] load metadata for docker.io/library/alpine:latest 0.5s
=> [alpine 1/2] FROM docker.io/library/alpine:latest@sha256:ceeae2849a425ef1a7e591d8288f1a58cdf1f4e8d9da7510e29ea829e61cf512 0.0s
=> => resolve docker.io/library/alpine:latest@sha256:ceeae2849a425ef1a7e591d8288f1a58cdf1f4e8d9da7510e29ea829e61cf512 0.0s
=> [internal] load build context 1.3s
=> => transferring context: 36.64MB 1.3s
=> [stage-1 1/4] FROM gcr.io/distroless/static:nonroot@sha256:80c956fb0836a17a565c43a4026c9c80b2013c83bea09f74fa4da195a59b7a99 0.0s
=> => resolve gcr.io/distroless/static:nonroot@sha256:80c956fb0836a17a565c43a4026c9c80b2013c83bea09f74fa4da195a59b7a99 0.0s
=> CACHED [alpine 2/2] RUN apk add -U --no-cache ca-certificates 0.0s
=> CACHED [stage-1 2/4] COPY --from=alpine /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ 0.0s
=> [stage-1 3/4] COPY /sentry /
amd64 上的构建
TBD
3.5 - 测试相关的构建学习
测试相关的构建学习
3.5.1 - Makefile tests部分
和测试相关的make target
https://github.com/dapr/dapr/blob/master/Makefile
################################################################################
# Target: tests #
################################################################################
include tests/dapr_tests.mk
https://github.com/dapr/dapr/blob/master/tests/dapr_tests.mk
变量定义
容器和k8s相关：
KUBECTL=kubectl
DAPR_CONTAINER_LOG_PATH?=./dist/container_logs
DAPR_TEST_SECONDARY_NAMESPACE=dapr-tests-2
# 测试用的 namespace，默认是 dapr-system，这个不合适，一般还是设置为 dapr-test 吧
ifeq ($(DAPR_TEST_NAMESPACE),)
DAPR_TEST_NAMESPACE=$(DAPR_NAMESPACE)
endif
ifeq ($(DAPR_TEST_REGISTRY),)
DAPR_TEST_REGISTRY=$(DAPR_REGISTRY)
endif
# 测试使用的镜像文件的 tag
ifeq ($(DAPR_TEST_TAG),)
DAPR_TEST_TAG=$(DAPR_TAG)-$(TARGET_OS)-$(TARGET_ARCH)
endif
# mimikube相关
ifeq ($(DAPR_TEST_ENV),minikube)
MINIKUBE_NODE_IP=$(shell minikube ip)
ifeq ($(MINIKUBE_NODE_IP),)
$(error cannot find get minikube node ip address. ensure that you have minikube environment.)
endif
endif
测试使用的 statue store 和 pub/sub 的默认值：
ifeq ($(DAPR_TEST_STATE_STORE),)
DAPR_TEST_STATE_STORE=redis
endif
ifeq ($(DAPR_TEST_QUERY_STATE_STORE),)
DAPR_TEST_QUERY_STATE_STORE=mongodb
endif
ifeq ($(DAPR_TEST_PUBSUB),)
DAPR_TEST_PUBSUB=redis
endif
检测当前os：
ifeq ($(OS),Windows_NT)
detected_OS := windows
else
detected_OS := $(shell sh -c 'uname 2>/dev/null || echo Unknown' | tr '[:upper:]' '[:lower:]')
endif
Target: get-components-contrib
################################################################################
# Target: get-components-contrib #
################################################################################
.PHONY: get-components-contrib
get-components-contrib:
go get github.com/dapr/components-contrib@master
获取 components-contrib 仓库 master 分支的最新代码。
执行结果输出如下：
$ make get-components-contrib
go get github.com/dapr/components-contrib@master
go: downloading github.com/dapr/components-contrib v1.6.0-rc.1.0.20220310012151-027204f2d3c5
go get: upgraded github.com/dapr/components-contrib v1.6.0-rc.1.0.20220307041340-f1209fb068c7 => v1.6.0-rc.1.0.20220310012151-027204f2d3c5
Target: create-test-namespace
create-test-namespace:
kubectl create namespace $(DAPR_TEST_NAMESPACE)
kubectl create namespace $(DAPR_TEST_SECONDARY_NAMESPACE)
Target: delete-test-namespace
delete-test-namespace:
kubectl delete namespace $(DAPR_TEST_NAMESPACE)
kubectl delete namespace $(DAPR_TEST_SECONDARY_NAMESPACE)
Target: setup-3rd-party
setup-3rd-party: setup-helm-init setup-test-env-redis setup-test-env-kafka setup-test-env-mongodb
Target: test-deps
.PHONY: test-deps
test-deps:
# The desire here is to download this test dependency without polluting go.mod
# In golang >=1.16 there is a new way to do this with `go install gotest.tools/gotestsum@latest`
# But this doesn't work with <=1.15.
# (see: https://golang.org/ref/mod#go-install)
command -v gotestsum || go install gotest.tools/gotestsum@latest
Target: setup-helm-init
# add required helm repo
setup-helm-init:
$(HELM) repo add bitnami https://charts.bitnami.com/bitnami
$(HELM) repo add stable https://charts.helm.sh/stable
$(HELM) repo add incubator https://charts.helm.sh/incubator
$(HELM) repo update
redis 相关的target
# install redis to the cluster without password
setup-test-env-redis:
$(HELM) install dapr-redis bitnami/redis --wait --timeout 5m0s --namespace $(DAPR_TEST_NAMESPACE) -f ./tests/config/redis_override.yaml
delete-test-env-redis:
${HELM} del dapr-redis --namespace ${DAPR_TEST_NAMESPACE}
kafka 相关的target
# install kafka to the cluster
setup-test-env-kafka:
$(HELM) install dapr-kafka bitnami/kafka -f ./tests/config/kafka_override.yaml --namespace $(DAPR_TEST_NAMESPACE) --timeout 10m0s
# delete kafka from cluster
delete-test-env-kafka:
$(HELM) del dapr-kafka --namespace $(DAPR_TEST_NAMESPACE)
mongodb 相关的target
# install mongodb to the cluster without password
setup-test-env-mongodb:
$(HELM) install dapr-mongodb bitnami/mongodb -f ./tests/config/mongodb_override.yaml --namespace $(DAPR_TEST_NAMESPACE) --wait --timeout 5m0s
# delete mongodb from cluster
delete-test-env-mongodb:
${HELM} del dapr-mongodb --namespace ${DAPR_TEST_NAMESPACE}
Target: setup-test-env
# Install redis and kafka to test cluster
setup-test-env: setup-test-env-kafka setup-test-env-redis setup-test-env-mongodb
保存k8s 资源和日志
save-dapr-control-plane-k8s-resources:
mkdir -p '$(DAPR_CONTAINER_LOG_PATH)'
kubectl describe all -n $(DAPR_TEST_NAMESPACE) > '$(DAPR_CONTAINER_LOG_PATH)/control_plane_k8s_resources.txt'
save-dapr-control-plane-k8s-logs:
mkdir -p '$(DAPR_CONTAINER_LOG_PATH)'
kubectl logs -l 'app.kubernetes.io/name=dapr' -n $(DAPR_TEST_NAMESPACE) > '$(DAPR_CONTAINER_LOG_PATH)/control_plane_containers.log'
Target: setup-disable-mtls
# Apply default config yaml to turn mTLS off for testing (mTLS is enabled by default)
setup-disable-mtls:
$(KUBECTL) apply -f ./tests/config/dapr_mtls_off_config.yaml --namespace $(DAPR_TEST_NAMESPACE)
Target: setup-app-configurations
# Apply default config yaml to turn tracing off for testing (tracing is enabled by default)
setup-app-configurations:
$(KUBECTL) apply -f ./tests/config/dapr_observability_test_config.yaml --namespace $(DAPR_TEST_NAMESPACE)
Target: setup-test-components
# Apply component yaml for state, secrets, pubsub, and bindings
setup-test-components: setup-app-configurations
$(KUBECTL) apply -f ./tests/config/kubernetes_secret.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/kubernetes_secret_config.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/kubernetes_redis_secret.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/dapr_$(DAPR_TEST_STATE_STORE)_state.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/dapr_$(DAPR_TEST_QUERY_STATE_STORE)_state.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/dapr_tests_cluster_role_binding.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/dapr_$(DAPR_TEST_PUBSUB)_pubsub.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/dapr_kafka_bindings.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/dapr_kafka_bindings_custom_route.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/dapr_kafka_bindings_grpc.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/app_topic_subscription_pubsub.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/app_topic_subscription_pubsub_grpc.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/kubernetes_allowlists_config.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/kubernetes_allowlists_grpc_config.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/dapr_redis_state_query.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/dapr_redis_state_badhost.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/dapr_redis_state_badpass.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/uppercase.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/pipeline.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/app_reentrant_actor.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/app_actor_type_metadata.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/app_topic_subscription_routing.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/app_topic_subscription_routing_grpc.yaml --namespace $(DAPR_TEST_NAMESPACE)
$(KUBECTL) apply -f ./tests/config/app_pubsub_routing.yaml --namespace $(DAPR_TEST_NAMESPACE)
# Show the installed components
$(KUBECTL) get components --namespace $(DAPR_TEST_NAMESPACE)
# Show the installed configurations
$(KUBECTL) get configurations --namespace $(DAPR_TEST_NAMESPACE)
Target: clean-test-env
# Clean up test environment
clean-test-env:
./tests/test-infra/clean_up.sh $(DAPR_TEST_NAMESPACE)
./tests/test-infra/clean_up.sh $(DAPR_TEST_NAMESPACE)-2
TODO: 这里应该用 DAPR_TEST_SECONDARY_NAMESPACE ， 随手 PR： fix build target clean-test-env by skyao · Pull Request #4420 · dapr/dapr (github.com)
kind 相关的 target
# Setup kind
setup-kind:
kind create cluster --config ./tests/config/kind.yaml --name kind
kubectl cluster-info --context kind-kind
# Setup registry
docker run -d --restart=always -p 5000:5000 --name kind-registry registry:2
# Connect the registry to the KinD network.
docker network connect "kind" kind-registry
# Setup metrics-server
helm install ms stable/metrics-server -n kube-system --set=args={--kubelet-insecure-tls}
describe-kind-env:
@echo "\
export MINIKUBE_NODE_IP=`kubectl get nodes \
-lkubernetes.io/hostname!=kind-control-plane \
-ojsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}'`\n\
export DAPR_REGISTRY=$${DAPR_REGISTRY:-localhost:5000/dapr}\n\
export DAPR_TEST_REGISTRY=$${DAPR_TEST_REGISTRY:-localhost:5000/dapr}\n\
export DAPR_TAG=dev\n\
export DAPR_NAMESPACE=dapr-tests"
delete-kind:
docker stop kind-registry && docker rm kind-registry || echo "Could not delete registry."
kind delete cluster --name kind
minikube 相关的 target
setup-minikube-darwin:
minikube start --memory=4g --cpus=4 --driver=hyperkit --kubernetes-version=v1.18.8
minikube addons enable metrics-server
setup-minikube-windows:
minikube start --memory=4g --cpus=4 --kubernetes-version=v1.18.8
minikube addons enable metrics-server
setup-minikube-linux:
minikube start --memory=4g --cpus=4 --kubernetes-version=v1.18.8
minikube addons enable metrics-server
setup-minikube: setup-minikube-$(detected_OS)
describe-minikube-env:
@echo "\
export DAPR_REGISTRY=docker.io/`docker-credential-desktop list | jq -r '\
. | to_entries[] | select(.key | contains("docker.io")) | last(.value)'`\n\
export DAPR_TAG=dev\n\
export DAPR_NAMESPACE=dapr-tests\n\
export DAPR_TEST_ENV=minikube\n\
export DAPR_TEST_REGISTRY=\n\
export MINIKUBE_NODE_IP="
# Setup minikube
delete-minikube:
minikube delete
3.5.2 - 端到端测试部分
和端到端测试相关的make target
E2e测试的 target 定义在 dapr_tests.mk：
https://github.com/dapr/dapr/blob/master/tests/dapr_tests.mk
但因为内容比较独立，单独拿出来细看
变量定义
E2E_TEST_APPS 定义了要 e2e 测试要用到的app：
# E2E test app list
# e.g. E2E_TEST_APPS=hellodapr state service_invocation
E2E_TEST_APPS=actorjava \
actordotnet \
actorpython \
actorphp \
hellodapr \
stateapp \
secretapp \
service_invocation \
service_invocation_grpc \
service_invocation_grpc_proxy_client \
service_invocation_grpc_proxy_server \
binding_input \
binding_input_grpc \
binding_output \
pubsub-publisher \
pubsub-subscriber \
pubsub-subscriber_grpc \
pubsub-subscriber-routing \
pubsub-subscriber-routing_grpc \
actorapp \
actorclientapp \
actorfeatures \
actorinvocationapp \
actorreentrancy \
runtime \
runtime_init \
middleware \
job-publisher \
E2E_TESTAPP_DIR 是 e2e 测试应用所在的目录：
# E2E test app root directory
E2E_TESTAPP_DIR=./tests/apps
Target: check-e2e-env
# check the required environment variables
check-e2e-env:
ifeq ($(DAPR_TEST_REGISTRY),)
$(error DAPR_TEST_REGISTRY environment variable must be set)
endif
ifeq ($(DAPR_TEST_TAG),)
$(error DAPR_TEST_TAG environment variable must be set)
endif
Target: build-e2e-app-x
define genTestAppImageBuild
.PHONY: build-e2e-app-$(1)
build-e2e-app-$(1): check-e2e-env
ifeq (,$(wildcard $(E2E_TESTAPP_DIR)/$(1)/$(DOCKERFILE)))
CGO_ENABLED=0 GOOS=$(TARGET_OS) GOARCH=$(TARGET_ARCH) go build -o $(E2E_TESTAPP_DIR)/$(1)/app$(BINARY_EXT_LOCAL) $(E2E_TESTAPP_DIR)/$(1)/app.go
$(DOCKER) build -f $(E2E_TESTAPP_DIR)/$(DOCKERFILE) $(E2E_TESTAPP_DIR)/$(1)/. -t $(DAPR_TEST_REGISTRY)/e2e-$(1):$(DAPR_TEST_TAG)
else
$(DOCKER) build -f $(E2E_TESTAPP_DIR)/$(1)/$(DOCKERFILE) $(E2E_TESTAPP_DIR)/$(1)/. -t $(DAPR_TEST_REGISTRY)/e2e-$(1):$(DAPR_TEST_TAG)
endif
endef
# Generate test app image build targets
$(foreach ITEM,$(E2E_TEST_APPS),$(eval $(call genTestAppImageBuild,$(ITEM))))
Target: push-e2e-app-x
define genTestAppImagePush
.PHONY: push-e2e-app-$(1)
push-e2e-app-$(1): check-e2e-env
$(DOCKER) push $(DAPR_TEST_REGISTRY)/e2e-$(1):$(DAPR_TEST_TAG)
endef
# Generate test app image push targets
$(foreach ITEM,$(E2E_TEST_APPS),$(eval $(call genTestAppImagePush,$(ITEM))))
Target: push-kind-e2e-app
define genTestAppImageKindPush
.PHONY: push-kind-e2e-app-$(1)
push-kind-e2e-app-$(1): check-e2e-env
kind load docker-image $(DAPR_TEST_REGISTRY)/e2e-$(1):$(DAPR_TEST_TAG)
endef
# Generate test app image push targets
$(foreach ITEM,$(E2E_TEST_APPS),$(eval $(call genTestAppImageKindPush,$(ITEM))))
Target: e2e-build-deploy-run
e2e-build-deploy-run: create-test-namespace setup-3rd-party build docker-push docker-deploy-k8s setup-test-components build-e2e-app-all push-e2e-app-all test-e2e-all
枚举 e2e 测试target列表
# Enumerate test app build targets
BUILD_E2E_APPS_TARGETS:=$(foreach ITEM,$(E2E_TEST_APPS),build-e2e-app-$(ITEM))
# Enumerate test app push targets
PUSH_E2E_APPS_TARGETS:=$(foreach ITEM,$(E2E_TEST_APPS),push-e2e-app-$(ITEM))
# Enumerate test app push targets
PUSH_KIND_E2E_APPS_TARGETS:=$(foreach ITEM,$(E2E_TEST_APPS),push-kind-e2e-app-$(ITEM))
Target：build-e2e-app-all
# build test app image
build-e2e-app-all: $(BUILD_E2E_APPS_TARGETS)
Target：push-e2e-app-all
# push test app image to the registry
push-e2e-app-all: $(PUSH_E2E_APPS_TARGETS)
Target：push-kind-e2e-app-all
# push test app image to kind cluster
push-kind-e2e-app-all: $(PUSH_KIND_E2E_APPS_TARGETS)
Target: test-e2e-all
# start all e2e tests
test-e2e-all: check-e2e-env test-deps
# Note: we can set -p 2 to run two tests apps at a time, because today we do not share state between
# tests. In the future, if we add any tests that modify global state (such as dapr config), we'll
# have to be sure and run them after the main test suite, so as not to alter the state of a running
# test
# Note2: use env variable DAPR_E2E_TEST to pick one e2e test to run.
ifeq ($(DAPR_E2E_TEST),)
DAPR_CONTAINER_LOG_PATH=$(DAPR_CONTAINER_LOG_PATH) GOOS=$(TARGET_OS_LOCAL) DAPR_TEST_NAMESPACE=$(DAPR_TEST_NAMESPACE) DAPR_TEST_TAG=$(DAPR_TEST_TAG) DAPR_TEST_REGISTRY=$(DAPR_TEST_REGISTRY) DAPR_TEST_MINIKUBE_IP=$(MINIKUBE_NODE_IP) gotestsum --jsonfile $(TEST_OUTPUT_FILE_PREFIX)_e2e.json --junitfile $(TEST_OUTPUT_FILE_PREFIX)_e2e.xml --format standard-quiet -- -p 2 -count=1 -v -tags=e2e ./tests/e2e/$(DAPR_E2E_TEST)/...
else
for app in $(DAPR_E2E_TEST); do \
DAPR_CONTAINER_LOG_PATH=$(DAPR_CONTAINER_LOG_PATH) GOOS=$(TARGET_OS_LOCAL) DAPR_TEST_NAMESPACE=$(DAPR_TEST_NAMESPACE) DAPR_TEST_TAG=$(DAPR_TEST_TAG) DAPR_TEST_REGISTRY=$(DAPR_TEST_REGISTRY) DAPR_TEST_MINIKUBE_IP=$(MINIKUBE_NODE_IP) gotestsum --jsonfile $(TEST_OUTPUT_FILE_PREFIX)_e2e.json --junitfile $(TEST_OUTPUT_FILE_PREFIX)_e2e.xml --format standard-quiet -- -p 2 -count=1 -v -tags=e2e ./tests/e2e/$$app/...; \
done
endif
3.5.3 - 性能测试部分
和性能测试相关的make target
性能测试的 target 定义在 dapr_tests.mk：
https://github.com/dapr/dapr/blob/master/tests/dapr_tests.mk
但因为内容比较独立，单独拿出来细看
变量定义
PERF_TEST_APPS 定义了性能测试要用到的app：
# PERFORMANCE test app list
PERF_TEST_APPS=actorfeatures actorjava tester service_invocation_http
PERF_TESTAPP_DIR 是性能测试应用所在的目录：
# PERFORMANCE test app root directory
PERF_TESTAPP_DIR=./tests/apps/perf
PERF_TESTS 是性能测试包含的测试案例，这些测试案例在 tests/perf 下：
# PERFORMANCE tests
PERF_TESTS=actor_timer actor_reminder actor_activation service_invocation_http
Target: build-perf-app-x
define genPerfTestAppImageBuild
.PHONY: build-perf-app-$(1)
build-perf-app-$(1): check-e2e-env
$(DOCKER) build -f $(PERF_TESTAPP_DIR)/$(1)/$(DOCKERFILE) $(PERF_TESTAPP_DIR)/$(1)/. -t $(DAPR_TEST_REGISTRY)/perf-$(1):$(DAPR_TEST_TAG)
endef
# Generate perf app image build targets
$(foreach ITEM,$(PERF_TEST_APPS),$(eval $(call genPerfTestAppImageBuild,$(ITEM))))
Target: perf-build-deploy-run
perf-build-deploy-run: create-test-namespace setup-3rd-party build docker-push docker-deploy-k8s setup-test-components build-perf-app-all push-perf-app-all test-perf-all
Target: push-perf-app-x
define genPerfAppImagePush
.PHONY: push-perf-app-$(1)
push-perf-app-$(1): check-e2e-env
$(DOCKER) push $(DAPR_TEST_REGISTRY)/perf-$(1):$(DAPR_TEST_TAG)
endef
# Generate perf app image push targets
$(foreach ITEM,$(PERF_TEST_APPS),$(eval $(call genPerfAppImagePush,$(ITEM))))
Target: push-kind-perf-app-x
define genPerfAppImageKindPush
.PHONY: push-kind-perf-app-$(1)
push-kind-perf-app-$(1): check-e2e-env
kind load docker-image $(DAPR_TEST_REGISTRY)/perf-$(1):$(DAPR_TEST_TAG)
endef
# Generate perf app image kind push targets
$(foreach ITEM,$(PERF_TEST_APPS),$(eval $(call genPerfAppImageKindPush,$(ITEM))))
枚举性能测试target列表
# Enumerate test app build targets
BUILD_PERF_APPS_TARGETS:=$(foreach ITEM,$(PERF_TEST_APPS),build-perf-app-$(ITEM))
# Enumerate perf app push targets
PUSH_PERF_APPS_TARGETS:=$(foreach ITEM,$(PERF_TEST_APPS),push-perf-app-$(ITEM))
# Enumerate perf app kind push targets
PUSH_KIND_PERF_APPS_TARGETS:=$(foreach ITEM,$(PERF_TEST_APPS),push-kind-perf-app-$(ITEM))
Target：build-perf-app-all
# build perf app image
build-perf-app-all: $(BUILD_PERF_APPS_TARGETS)
Target：push-perf-app-all
# push perf app image to the registry
push-perf-app-all: $(PUSH_PERF_APPS_TARGETS)
Target：push-kind-perf-app-all
# push perf app image to kind cluster
push-kind-perf-app-all: $(PUSH_KIND_PERF_APPS_TARGETS)
Target: test-perf-x
define genPerfTestRun
.PHONY: test-perf-$(1)
test-perf-$(1): check-e2e-env test-deps
DAPR_CONTAINER_LOG_PATH=$(DAPR_CONTAINER_LOG_PATH) GOOS=$(TARGET_OS_LOCAL) DAPR_TEST_NAMESPACE=$(DAPR_TEST_NAMESPACE) DAPR_TEST_TAG=$(DAPR_TEST_TAG) DAPR_TEST_REGISTRY=$(DAPR_TEST_REGISTRY) DAPR_TEST_MINIKUBE_IP=$(MINIKUBE_NODE_IP) gotestsum --jsonfile $(TEST_OUTPUT_FILE_PREFIX)_perf_$(1).json --junitfile $(TEST_OUTPUT_FILE_PREFIX)_perf_$(1).xml --format standard-quiet -- -timeout 1h -p 1 -count=1 -v -tags=perf ./tests/perf/$(1)/...
jq -r .Output $(TEST_OUTPUT_FILE_PREFIX)_perf_$(1).json | strings
endef
# Generate perf app image build targets
$(foreach ITEM,$(PERF_TESTS),$(eval $(call genPerfTestRun,$(ITEM))))
TEST_PERF_TARGETS:=$(foreach ITEM,$(PERF_TESTS),test-perf-$(ITEM))
Target: test-perf-all
# start all perf tests
test-perf-all: check-e2e-env test-deps
DAPR_CONTAINER_LOG_PATH=$(DAPR_CONTAINER_LOG_PATH) GOOS=$(TARGET_OS_LOCAL) DAPR_TEST_NAMESPACE=$(DAPR_TEST_NAMESPACE) DAPR_TEST_TAG=$(DAPR_TEST_TAG) DAPR_TEST_REGISTRY=$(DAPR_TEST_REGISTRY) DAPR_TEST_MINIKUBE_IP=$(MINIKUBE_NODE_IP) gotestsum --jsonfile $(TEST_OUTPUT_FILE_PREFIX)_perf.json --junitfile $(TEST_OUTPUT_FILE_PREFIX)_perf.xml --format standard-quiet -- -p 1 -count=1 -v -tags=perf ./tests/perf/...
jq -r .Output $(TEST_OUTPUT_FILE_PREFIX)_perf.json | strings
m1上执行
执行结果输出如下：
$ make test-perf-all
# The desire here is to download this test dependency without polluting go.mod
# In golang >=1.16 there is a new way to do this with `go install gotest.tools/gotestsum@latest`
# But this doesn't work with <=1.15.
# (see: https://golang.org/ref/mod#go-install)
command -v gotestsum || go install gotest.tools/gotestsum@latest
/Users/sky/work/soft/gopath/bin/gotestsum
DAPR_CONTAINER_LOG_PATH=./dist/container_logs GOOS=darwin DAPR_TEST_NAMESPACE=dapr-system DAPR_TEST_TAG=dev-linux-arm64 DAPR_TEST_REGISTRY=docker.io/skyao DAPR_TEST_MINIKUBE_IP= gotestsum --jsonfile ./test_report_perf.json --junitfile ./test_report_perf.xml --format standard-quiet -- -p 1 -count=1 -v -tags=perf ./tests/perf/...
? github.com/dapr/dapr/tests/perf [no test files]
2022/03/24 10:52:06 Running setup...
2022/03/24 10:52:06 Installing test apps...
2022/03/24 10:52:06 Adding app {testapp 3000 map[] true perf-actorjava:dev-linux-arm64 docker.io/skyao 1 true true 4.0 0.1 800Mi 2500Mi 4.0 0.1 512Mi 250Mi <nil> false}
2022/03/24 10:52:06 Adding app {tester 3001 map[] true perf-tester:dev-linux-arm64 docker.io/skyao 1 true true 4.0 0.1 800Mi 2500Mi 4.0 0.1 512Mi 250Mi <nil> false}
2022/03/24 10:52:06 Installing apps ...
2022/03/24 10:52:08 Deploying app testapp ...
2022/03/24 11:02:10 deployment testapp relate pods: ......
CST,LastTransitionTime:2022-03-24 11:02:10 +0800 CST,},},ReadyReplicas:0,CollisionCount:nil,},} pod status: map[testapp-7cc8965d47-fzqhs:[]] error: timed out waiting for the condition2022/03/24 11:02:10 Running teardown...
FAIL github.com/dapr/dapr/tests/perf/actor_activation 605.705s
4.1 - quickstarts 简介
quickstarts 仓库存放dapr 的 quickstart 内容
https://github.com/dapr/quickstarts
quickstarts 仓库存放dapr 的 quickstart 内容。
背景：
- quickstarts （或者说 Tutorials）内容之前是存放在各个仓库的
- 后续被统一存放到这个 quickstart 仓库中
- 内部不仅仅包括各个构建块，也包括各种语言的sdk
- 因此项目内混合有多种语言的项目
这个仓库的内容是配合 Dapr Quickstarts 文档的。
4.2 - quickstarts 构建
quickstarts 仓库的构建
由于 quickstarts 仓库存放有各种语言的不同项目，因此它的构建是和 quickstart 都不一样的。
详细构建方式参考 https://skyao.io/learning-dapr/docs/introduction/quickstart/
5.1 - 安装配置hugo
准备 hugo 以便构建文档
准备工作
安装golang
安装 node
ubuntu下：
wget https://nodejs.org/dist/v18.18.0/node-v18.18.0-linux-x64.tar.xz
tar xvf node-v18.18.0-linux-x64.tar.xz
sudo mv node-v18.18.0-linux-x64 /usr/share/nodejs
然后修改
vi ~/.zshrc
加入:
export PATH=/usr/share/nodejs/bin:$PATH
生效并检查：
source ~/.zshrc
npm --version
安装hugo
wget https://github.com/gohugoio/hugo/releases/download/v0.119.0/hugo_extended_0.119.0_linux-amd64.deb
sudo dpkg -i hugo_extended_0.119.0_linux-amd64.deb
5.2 - 构建daprdocs的hugo文档
构建 daprdocs 的hugo文档
准备工作
更新 git submodules
cd ./daprdocs
git submodule update --init --recursive
安装 npm 包
npm install
构建daprdocs项目
在本机构建：
hugo server --disableFastRender
然后访问 http://localhost:1313/
如果不是在本机构建，则需要修改 bind 地址：
hugo server --baseURL https://192.168.99.100:1313 --bind="0.0.0.0" --disableFastRender
然后通过 http://192.168.99.100:1313/ 进行访问
6.1 - validate脚本
java-sdk仓库 validate 脚本
name：Auto Validate Examples
https://github.com/dapr/java-sdk/actions/workflows/validate.yml
脚本源文件：
https://github.com/dapr/java-sdk/blob/master/.github/workflows/validate.yml
on
适用于 push 和 PR：
on:
workflow_dispatch:
push:
branches:
- master
- release-*
tags:
- v*
pull_request:
branches:
- master
- release-*
Job 详细
jobs:
validate:
runs-on: ubuntu-latest
strategy:
fail-fast: false # Keep running if one leg fails.
matrix:
java: [ 11, 13, 15, 16 ]
Java matrix 包含 11 / 13 / 15 / 16 等多个 Java 版本，
https://github.com/dapr/java-sdk/actions/runs/7492324406/job/20395558712?pr=987
环境变量准备
env:
GOVER: "1.20"
GOOS: linux
GOARCH: amd64
GOPROXY: https://proxy.golang.org
JDK_VER: ${{ matrix.java }}
DAPR_CLI_VER: 1.12.0-rc.1
DAPR_RUNTIME_VER: 1.12.0-rc.5
DAPR_INSTALL_URL: https://raw.githubusercontent.com/dapr/cli/v1.12.0-rc.1/install/install.sh
DAPR_CLI_REF:
DAPR_REF:
步骤
前期准备工作
actions/checkout
- uses: actions/checkout@v4
Set up OpenJDK
- name: Set up OpenJDK ${{ env.JDK_VER }}
uses: actions/setup-java@v4
with:
distribution: 'adopt'
java-version: ${{ env.JDK_VER }}
其中 JDK_VER 来源于 matrix.java：
JDK_VER: ${{ matrix.java }}
Set up Dapr CLI
- name: Set up Dapr CLI
run: wget -q ${{ env.DAPR_INSTALL_URL }} -O - | /bin/bash -s ${{ env.DAPR_CLI_VER }}
Set up Go
- name: Set up Go ${{ env.GOVER }}
if: env.DAPR_REF != '' || env.DAPR_CLI_REF != ''
uses: actions/setup-go@v5
with:
go-version: ${{ env.GOVER }}
Checkout Dapr CLI repo to override dapr command.
- name: Checkout Dapr CLI repo to override dapr command.
uses: actions/checkout@v4
if: env.DAPR_CLI_REF != ''
with:
repository: dapr/cli
ref: ${{ env.DAPR_CLI_REF }}
path: cli
Checkout Dapr repo to override daprd.
- name: Checkout Dapr repo to override daprd.
uses: actions/checkout@v4
if: env.DAPR_REF != ''
with:
repository: dapr/dapr
ref: ${{ env.DAPR_REF }}
path: dapr
Build and override dapr cli with referenced commit
- name: Build and override dapr cli with referenced commit.
if: env.DAPR_CLI_REF != ''
run: |
cd cli
make
sudo cp dist/linux_amd64/release/dapr /usr/local/bin/dapr
cd ..
Initialize Dapr runtime
- name: Initialize Dapr runtime ${{ env.DAPR_RUNTIME_VER }}
run: |
dapr uninstall --all
dapr init --runtime-version ${{ env.DAPR_RUNTIME_VER }}
Build and override daprd with referenced commit
- name: Build and override daprd with referenced commit.
if: env.DAPR_REF != ''
run: |
cd dapr
make
mkdir -p $HOME/.dapr/bin/
cp dist/linux_amd64/release/daprd $HOME/.dapr/bin/daprd
cd ..
Override placement service
- name: Override placement service.
if: env.DAPR_REF != ''
run: |
docker stop dapr_placement
cd dapr
./dist/linux_amd64/release/placement &
Install utilities dependencies
- name: Install utilities dependencies
run: |
echo "PATH=$PATH:$HOME/.local/bin" >> $GITHUB_ENV
pip3 install setuptools wheel
pip3 install mechanical-markdown
Install Local mongo database using docker-compose
- name: Install Local mongo database using docker-compose
run: |
docker-compose -f ./sdk-tests/deploy/local-test.yml up -d mongo
docker ps
构建 java SDK
Clean up files
- name: Clean up files
run: ./mvnw clean
Build sdk
- name: Build sdk
run: ./mvnw compile -q
Install jars
- name: Install jars
run: ./mvnw install -q
Validate invoke http example
- name: Validate invoke http example
working-directory: ./examples
run: |
mm.py ./src/main/java/io/dapr/examples/invoke/http/README.md
debug错误
./mvnw install -q
shell: /usr/bin/bash -e {0}
env:
GOVER: 1.20
GOOS: linux
GOARCH: amd64
GOPROXY: https://proxy.golang.org
JDK_VER: 11
DAPR_CLI_VER: 1.12.0-rc.1
DAPR_RUNTIME_VER: 1.12.0-rc.5
DAPR_INSTALL_URL: https://raw.githubusercontent.com/dapr/cli/v1.12.0-rc.1/install/install.sh
DAPR_CLI_REF:
DAPR_REF:
JAVA_HOME: /opt/hostedtoolcache/Java_Adopt_jdk/11.0.21-9/x64
JAVA_HOME_11_X64: /opt/hostedtoolcache/Java_Adopt_jdk/11.0.21-9/x64
PATH: /opt/hostedtoolcache/Java_Adopt_jdk/11.0.21-9/x64/bin:/snap/bin:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/runner/.local/bin
./mvnw install -q
shell: /usr/bin/bash -e {0}
env:
GOVER: 1.20
GOOS: linux
GOARCH: amd64
GOPROXY: https://proxy.golang.org
JDK_VER: 17
DAPR_CLI_VER: 1.12.0-rc.1
DAPR_RUNTIME_VER: 1.12.0-rc.5
DAPR_INSTALL_URL: https://raw.githubusercontent.com/dapr/cli/v1.12.0-rc.1/install/install.sh
DAPR_CLI_REF:
DAPR_REF:
JAVA_HOME: /opt/hostedtoolcache/Java_Adopt_jdk/17.0.9-9/x64
JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Adopt_jdk/17.0.9-9/x64
PATH: /opt/hostedtoolcache/Java_Adopt_jdk/17.0.9-9/x64/bin:/snap/bin:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/runner/.local/bin
Error: Error fetching link: /home/runner/work/java-sdk/java-sdk/sdk-autogen/target/javadoc-bundle-options. Ignored it.
== APP == [INFO] --- exec:3.1.1:java (default-cli) @ dapr-sdk-tests ---
== APP == [WARNING]
== APP == java.lang.UnsupportedClassVersionError: io/dapr/it/actors/services/springboot/DemoActorService has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0
== APP == at java.lang.ClassLoader.defineClass1 (Native Method)
== APP == at java.lang.ClassLoader.defineClass (ClassLoader.java:1022)
== APP == at java.security.SecureClassLoader.defineClass (SecureClassLoader.java:174)
== APP == at java.net.URLClassLoader.defineClass (URLClassLoader.java:555)
== APP == at java.net.URLClassLoader$1.run (URLClassLoader.java:458)
== APP == at java.net.URLClassLoader$1.run (URLClassLoader.java:452)
== APP == at java.security.AccessController.doPrivileged (Native Method)
== APP == at java.net.URLClassLoader.findClass (URLClassLoader.java:451)
== APP == at org.codehaus.mojo.exec.URLClassLoaderBuilder$ExecJavaClassLoader.loadClass (URLClassLoaderBuilder.java:138)
== APP == at java.lang.ClassLoader.loadClass (ClassLoader.java:527)
== APP == at org.codehaus.mojo.exec.ExecJavaMojo.lambda$execute$0 (ExecJavaMojo.java:273)
== APP == at java.lang.Thread.run (Thread.java:829)
== APP == [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.1.1:java (default-cli) on project dapr-sdk-tests: An exception occurred while executing the Java class. io/dapr/it/actors/services/springboot/DemoActorService has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 -> [Help 1]
Started a new sub-workflow model workflow with instance ID: bb8fe22c-d76b-4db1-ae77-51981ead07e2
workflow instance with ID: bb8fe22c-d76b-4db1-ae77-51981ead07e2 completed with result: !wolfkroW rpaD olleH
ERROR expected lines not found:
calling subworkflow with input: Hello Dapr Workflow!
SubWorkflow finished with: !wolfkroW rpaD olleH
Expected stderr (output_match_mode: substring, match_order: none):
Actual stderr: