Kubebuilder 介绍及使用

架构概念图

控制器

使用

前提

部署 k8s 集群

安装 kubelet 客户端程序

安装 golang

创建项目

初始化


kubebuilder init --domain=ljtian.com --repo "github.com/ljtian/operator-demo"

输出内容


INFO Writing kustomize manifests for you to edit... 
INFO Writing scaffold for you to edit...          
INFO Get controller runtime:
$ go get sigs.k8s.io/controller-runtime@v0.20.4 
INFO Update dependencies:
$ go mod tidy           
go: downloading google.golang.org/genproto v0.0.0-20240123012728-ef4313101c80
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.16.0
protocol error: received DATA after END_STREAM
Next: define a resource with:
$ kubebuilder create api

创建 API


kubebuilder create api --group cache --version v1alpha1 --kind Memcached

输出内容


INFO Create Resource [y/n]                        
y
INFO Create Controller [y/n]                      
y
INFO Writing kustomize manifests for you to edit... 
INFO Writing scaffold for you to edit...          
INFO api/v1alpha1/memcached_types.go              
INFO api/v1alpha1/groupversion_info.go            
INFO internal/controller/suite_test.go            
INFO internal/controller/memcached_controller.go  
INFO internal/controller/memcached_controller_test.go 
INFO Update dependencies:
$ go mod tidy           
INFO Running make:
$ make generate                
mkdir -p /home/ljtian/git/operator-demo/bin
Downloading sigs.k8s.io/controller-tools/cmd/controller-gen@v0.17.2
go: downloading sigs.k8s.io/controller-tools v0.17.2
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading golang.org/x/tools v0.29.0
go: downloading github.com/fatih/color v1.18.0
go: downloading github.com/gobuffalo/flect v1.0.3
go: downloading github.com/spf13/pflag v1.0.6
go: downloading golang.org/x/net v0.34.0
go: downloading github.com/mattn/go-isatty v0.0.20
go: downloading github.com/mattn/go-colorable v0.1.13
go: downloading golang.org/x/sys v0.29.0
go: downloading golang.org/x/text v0.21.0
go: downloading golang.org/x/mod v0.22.0
go: downloading golang.org/x/sync v0.10.0
/home/ljtian/git/operator-demo/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
Next: implement your new API and generate the manifests (e.g. CRDs,CRs) with:
$ make manifests

查看当前目录状态


├── api
│   └── v1alpha1
│       ├── groupversion_info.go
│       ├── memcached_types.go
│       └── zz_generated.deepcopy.go
├── bin
│   ├── controller-gen -> /home/ljtian/git/operator-demo/bin/controller-gen-v0.17.2
│   └── controller-gen-v0.17.2
├── cmd
│   └── main.go
├── config
│   ├── crd
│   │   ├── kustomization.yaml
│   │   └── kustomizeconfig.yaml
│   ├── default
│   │   ├── cert_metrics_manager_patch.yaml
│   │   ├── kustomization.yaml
│   │   ├── manager_metrics_patch.yaml
│   │   └── metrics_service.yaml
│   ├── manager
│   │   ├── kustomization.yaml
│   │   └── manager.yaml
│   ├── network-policy
│   │   ├── allow-metrics-traffic.yaml
│   │   └── kustomization.yaml
│   ├── prometheus
│   │   ├── kustomization.yaml
│   │   ├── monitor_tls_patch.yaml
│   │   └── monitor.yaml
│   ├── rbac
│   │   ├── kustomization.yaml
│   │   ├── leader_election_role_binding.yaml
│   │   ├── leader_election_role.yaml
│   │   ├── memcached_admin_role.yaml
│   │   ├── memcached_editor_role.yaml
│   │   ├── memcached_viewer_role.yaml
│   │   ├── metrics_auth_role_binding.yaml
│   │   ├── metrics_auth_role.yaml
│   │   ├── metrics_reader_role.yaml
│   │   ├── role_binding.yaml
│   │   ├── role.yaml
│   │   └── service_account.yaml
│   └── samples
│       ├── cache_v1alpha1_memcached.yaml
│       └── kustomization.yaml
├── Dockerfile
├── go.mod
├── go.sum
├── hack
│   └── boilerplate.go.txt
├── internal
│   └── controller
│       ├── memcached_controller.go
│       ├── memcached_controller_test.go
│       └── suite_test.go
├── Makefile
├── PROJECT
├── README.md
└── test
    ├── e2e
    │   ├── e2e_suite_test.go
    │   └── e2e_test.go
    └── utils
        └── utils.go

上面的是 kubebuilder 生成的项目脚手架，需要修改及关注的主要文件为：

api/v1alpha1/memcached_types.go: CRD 类型定义位置

internal/controller/memcached_controller.go: 对账逻辑编写位置

修改代码文件(memcached_types.go)


// MemcachedSpec defines the desired state of Memcached.
type MemcachedSpec struct {
    // 插入额外的规格字段 - 集群的期望状态
    // 注意：修改此文件后，运行“make”以重新生成代码

    // Size defines the number of Memcached instances
    // The following markers will use OpenAPI v3 schema to validate the value
    // More info: https://book.kubebuilder.io/reference/markers/crd-validation.html
    // +kubebuilder:validation:Minimum=1
    // +kubebuilder:validation:Maximum=3
    // +kubebuilder:validation:ExclusiveMaximum=false
    Size int32 `json:"size,omitempty"`
}

// MemcachedStatus defines the observed state of Memcached.
type MemcachedStatus struct {
    // Represents the observations of a Memcached's current state.
    // Memcached.status.conditions.type are: "Available", "Progressing", and "Degraded"
    // Memcached.status.conditions.status are one of True, False, Unknown.
    // Memcached.status.conditions.reason the value should be a CamelCase string and producers of specific
    // condition types may define expected values and meanings for this field, and whether the values
    // are considered a guaranteed API.
    // Memcached.status.conditions.Message is a human readable message indicating details about the transition.
    // For further information see: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties

    Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"`
}

// +kubebuilder:object:root=true
// +kubebuilder:subresource:status

// Memcached is the Schema for the memcacheds API.
type Memcached struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`

    Spec   MemcachedSpec   `json:"spec,omitempty"`
    Status MemcachedStatus `json:"status,omitempty"`
}

// +kubebuilder:object:root=true

// MemcachedList contains a list of Memcached.
type MemcachedList struct {
    metav1.TypeMeta `json:",inline"`
    metav1.ListMeta `json:"metadata,omitempty"`
    Items           []Memcached `json:"items"`
}

func init() {
    SchemeBuilder.Register(&Memcached{}, &MemcachedList{})
}

修改的内容请查看[https://github.com/LJTian/operator-demo/commit/3e43accdb7082dff7753378dd1da8c45307001b9]

生成具有规格和验证的清单


root@localhost # make generate
/home/ljtian/git/operator-demo/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
root@localhost # make manifests
/home/ljtian/git/operator-demo/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases

生成所有所需的文件：

运行 make generate,会在 api/v1alpha1/zz_generated.deepcopy.go 文件中添加深度拷贝（DeepCopy）内容。

运行 make manifests,会在 config/crd/bases 下生成 CRD 清单，并在 config/crd/samples 下生成一个示例文件。

问题记录

在 GOPATH 之外目录创建，需要使用 —repo 参数。错误现象如下：


Error: failed to initialize project: unable to inject the configuration to "base.go.kubebuilder.io/v4": error finding current repository: could not determine repository path from module data, package data, or by initializing a module: go: cannot determine module path for source directory /home/ljtian/git/operator-demo (outside GOPATH, module path must be specified)

Example usage:
        'go mod init example.com/m' to initialize a v0 or v1 module
        'go mod init example.com/m/v2' to initialize a v2 module

Run 'go help mod init' for more information.

Usage:
  kubebuilder init [flags]

Examples:
  # Initialize a new project with your domain and name in copyright
  kubebuilder init --plugins go/v4 --domain example.org --owner "Your name"

  # Initialize a new project defining a specific project version
  kubebuilder init --plugins go/v4 --project-version 3


Flags:
      --domain string            domain for groups (default "my.domain")
      --fetch-deps               ensure dependencies are downloaded (default true)
  -h, --help                     help for init
      --license string           license to use to boilerplate, may be one of 'apache2', 'none' (default "apache2")
      --owner string             owner to add to the copyright
      --project-name string      name of this project
      --project-version string   project version (default "3")
      --repo string              name to use for go module (e.g., github.com/user/repo), defaults to the go package of the current working directory.
      --skip-go-version-check    if specified, skip checking the Go version

Global Flags:
      --plugins strings   plugin keys to be used for this subcommand execution

FATA failed to initialize project: unable to inject the configuration to "base.go.kubebuilder.io/v4": error finding current repository: could not determine repository path from module data, package data, or by initializing a module: go: cannot determine module path for source directory /home/ljtian/git/operator-demo (outside GOPATH, module path must be specified)

Example usage:
        'go mod init example.com/m' to initialize a v0 or v1 module
        'go mod init example.com/m/v2' to initialize a v2 module

Run 'go help mod init' for more information.

没有设置 GO 代理，导致拉取模块失败


INFO Writing kustomize manifests for you to edit... 
INFO Writing scaffold for you to edit...          
INFO Get controller runtime:
$ go get sigs.k8s.io/controller-runtime@v0.20.4 
go: sigs.k8s.io/controller-runtime@v0.20.4: Get "https://proxy.golang.org/sigs.k8s.io/controller-runtime/@v/v0.20.4.mod": dial tcp 142.251.211.241:443: i/o timeout
Error: failed to initialize project: unable to scaffold with "base.go.kubebuilder.io/v4": exit status 1
Usage:
  kubebuilder init [flags]

Examples:
  # Initialize a new project with your domain and name in copyright
  kubebuilder init --plugins go/v4 --domain example.org --owner "Your name"

  # Initialize a new project defining a specific project version
  kubebuilder init --plugins go/v4 --project-version 3


Flags:
      --domain string            domain for groups (default "my.domain")
      --fetch-deps               ensure dependencies are downloaded (default true)
  -h, --help                     help for init
      --license string           license to use to boilerplate, may be one of 'apache2', 'none' (default "apache2")
      --owner string             owner to add to the copyright
      --project-name string      name of this project
      --project-version string   project version (default "3")
      --repo string              name to use for go module (e.g., github.com/user/repo), defaults to the go package of the current working directory.
      --skip-go-version-check    if specified, skip checking the Go version

Global Flags:
      --plugins strings   plugin keys to be used for this subcommand execution

FATA failed to initialize project: unable to scaffold with "base.go.kubebuilder.io/v4": exit status 1

处理方式


echo "export GO111MODULE=on" >> ~/.profile
echo "export GOPROXY=https://goproxy.cn" >> ~/.profile
source ~/.profile