发表更新23 分钟读完 (大约3433个字)

Kubernetes APIServer API Resource Installation

在Kubernetes APIServer Storage 框架解析中，我们介绍了APIServer相关的存储框架，每个API资源，都有对应的REST store以及etcd store。在Kubernetes APIServer GenericAPIServer中介绍了GenericAPIServer的Handler是如何构建，API对象是如何以APIGroupInfo的形式注册进Handler中的。在Kubernetes APIServer 机制概述中简单介绍了APIServer的扩展机制，即Aggregator, APIExtensions以及KubeAPIServer这三者之间通过Delegation的方式实现了扩展。本篇文章就重点介绍下这三个”扩展对象”中的API对象资源是如何组织成APIGroupInfo的，然后怎么调用GenericAPIServer中暴露出来的安装方法进行安装的，最后盘点下当前版本的Kubernetes中，都有哪些API对象资源。

KubeAPIServer是Kubernetes内置的API对象所在的APIServer，而Aggregator和APIExtensions是Kubernetes API的两个扩展机制，对这两个扩展机制的介绍见官方文档，APIExtensions就是CRD的实现，而Aggregator是一种高级扩展，可以让Kubernetes APIServer跟外部的APIServer进行联动，这三者中，每个都包含一个GenericAPIServer，先来看下这三个对应的结构体：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
# kubernetes/pkg/controlplane/instance.go

// KubeAPIServer
type Instance struct {
    GenericAPIServer *genericapiserver.GenericAPIServer

    ClusterAuthenticationInfo clusterauthenticationtrust.ClusterAuthenticationInfo
}

# kube-aggregator/pkg/apiserver/apiserver.go

// Aggregator
type APIAggregator struct {
    GenericAPIServer *genericapiserver.GenericAPIServer

    delegateHandler http.Handler

    // proxyClientCert/Key are the client cert used to identify this proxy. Backing APIServices use
    // this to confirm the proxy's identity
    proxyClientCert []byte
    proxyClientKey  []byte
    proxyTransport  *http.Transport

    // proxyHandlers are the proxy handlers that are currently registered, keyed by apiservice.name
    proxyHandlers map[string]*proxyHandler
    // handledGroups are the groups that already have routes
    handledGroups sets.String

    ......
}

# apiextensions-apiserver/pkg/apiserver/apiserver.go

// APIExtensions
type CustomResourceDefinitions struct {
    GenericAPIServer *genericapiserver.GenericAPIServer

    // provided for easier embedding
    Informers externalinformers.SharedInformerFactory
}

他们各自的API对象都是安装注册到各自的GenericAPIServer中的，除了Instance中Kubernetes API内置的像pods, services这些API对象外，APIAggregator和CustomResourceDefinitions也都内置了各自的API对象，不过这些API对象也都是为了本身的扩展而设计的，APIAggregator中内置的API对象叫做apiservices，所属的组为apiregistration.k8s.io，每一个外部的APIServer都抽象为这个apiservices，注册到APIAggregator中，而apiextensions中内置的API对象就叫做customresourcedefinations，所属的组为apiextensions.k8s.io，这就是我们常说的CRD了，每一个自定义的资源，都抽象为一个CRD。

注意，这里面的名词，KubeAPIServer和Instance对应，Aggretator和APIAggregator对应，APIExtensions和CustomResourceDefinitions对应，前者是在代码中他们各自的GenericAPIServer的name，而后者是对应的结构体的名字。

实例化上面的三个结构体，就是在Kubernetes APIServer 机制概述介绍的CreateServerChain()阶段做的，通过Config->Complete->New模式被初始化出来，核心的逻辑，在New()方法中，我们重点关注下其中的安装API对象的逻辑，来分别看下。

KubeAPIServer

KubeAPIServer中内置的对象分为两类，一类是Legacy的，是早期设计的API，那时候还没有分组的设计，它里面API对象的前缀统一是这样的: /api/v1，像pods, services, nodes都属于这一类，路径中不带组信息，一般我们称他们为core/legacy组，另一类就是有分组设计的了，它里面API对象的前缀都是带组和版本信息的: /apis/$GROUP_NAME/$VERSION，像deployments, daemonsets都属于这一类的，这种我们称之为named group。这两种API，在Instace的New()方法中，有不同的组织方式：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

# kubernetes/pkg/controlplane/instance.go

func (c completedConfig) New(delegationTarget genericapiserver.DelegationTarget) (*Instance, error) {

    s, err := c.GenericConfig.New("kube-apiserver", delegationTarget)

    ......

    m := &Instance{
        GenericAPIServer:          s,
        ClusterAuthenticationInfo: c.ExtraConfig.ClusterAuthenticationInfo,
    }
    ......

    legacyRESTStorageProvider, err := corerest.New(corerest.Config{
        GenericConfig: corerest.GenericConfig{
            StorageFactory:              c.ExtraConfig.StorageFactory,
            EventTTL:                    c.ExtraConfig.EventTTL,
            LoopbackClientConfig:        c.GenericConfig.LoopbackClientConfig,
            ServiceAccountIssuer:        c.ExtraConfig.ServiceAccountIssuer,
            ExtendExpiration:            c.ExtraConfig.ExtendExpiration,
            ServiceAccountMaxExpiration: c.ExtraConfig.ServiceAccountMaxExpiration,
            APIAudiences:                c.GenericConfig.Authentication.APIAudiences,
            Informers:                   c.ExtraConfig.VersionedInformers,
        },
        Proxy: corerest.ProxyConfig{
            Transport:           c.ExtraConfig.ProxyTransport,
            KubeletClientConfig: c.ExtraConfig.KubeletClientConfig,
        },
        Services: corerest.ServicesConfig{
            ClusterIPRange:          c.ExtraConfig.ServiceIPRange,
            SecondaryClusterIPRange: c.ExtraConfig.SecondaryServiceIPRange,
            NodePortRange:           c.ExtraConfig.ServiceNodePortRange,
            IPRepairInterval:        c.ExtraConfig.RepairServicesInterval,
        },
    })

    restStorageProviders := []RESTStorageProvider{
        legacyRESTStorageProvider,
        apiserverinternalrest.StorageProvider{},
        authenticationrest.RESTStorageProvider{Authenticator: c.GenericConfig.Authentication.Authenticator, APIAudiences: c.GenericConfig.Authentication.APIAudiences},
        authorizationrest.RESTStorageProvider{Authorizer: c.GenericConfig.Authorization.Authorizer, RuleResolver: c.GenericConfig.RuleResolver},
        autoscalingrest.RESTStorageProvider{},
        batchrest.RESTStorageProvider{},
        certificatesrest.RESTStorageProvider{},
        coordinationrest.RESTStorageProvider{},
        discoveryrest.StorageProvider{},
        networkingrest.RESTStorageProvider{},
        noderest.RESTStorageProvider{},
        policyrest.RESTStorageProvider{},
        rbacrest.RESTStorageProvider{Authorizer: c.GenericConfig.Authorization.Authorizer},
        schedulingrest.RESTStorageProvider{},
        storagerest.RESTStorageProvider{},
        flowcontrolrest.RESTStorageProvider{InformerFactory: c.GenericConfig.SharedInformerFactory},
        // keep apps after extensions so legacy clients resolve the extensions versions of shared resource names.
        // See https://github.com/kubernetes/kubernetes/issues/42392
        appsrest.StorageProvider{},
        admissionregistrationrest.RESTStorageProvider{Authorizer: c.GenericConfig.Authorization.Authorizer, DiscoveryClient: discoveryClientForAdmissionRegistration},
        eventsrest.RESTStorageProvider{TTL: c.ExtraConfig.EventTTL},
        resourcerest.RESTStorageProvider{},
    }
    if err := m.InstallAPIs(c.ExtraConfig.APIResourceConfigSource, c.GenericConfig.RESTOptionsGetter, restStorageProviders...); err != nil {
        return nil, err
    }

    ......
}

可以看到，针对每个group，构造了一个RESTStorageProvider结构体，包括core group也是，这些结构体都实现了下面的接口：

1
2
3
4
5
6
# kubernetes/pkg/controlplane/instance.go

type RESTStorageProvider interface {
    GroupName() string
    NewRESTStorage(apiResourceConfigSource serverstorage.APIResourceConfigSource, restOptionsGetter generic.RESTOptionsGetter) (genericapiserver.APIGroupInfo, error)
}

NewRESTStorage() 方法很重要，它的主要作用就是构建某个Group的各种版本的各种资源的REST store，将其组装成前面介绍过的 APIGroupInfo 结构体，它传了两个参数：

  • apiResourceConfigSource 保存了某个版本(GroupVersion)或者资源(GroupVersionResource)是否要启用的开关，因为Kubernetes的API是多版本的API，会有多个版本共存，但是并不是所有版本的API都会启用，默认只启用稳定版本的API，还有一些因为历史遗留问题而需要默认开启的beta版本的API，可以通过 --runtime-config 来配置开启哪些版本或者资源，但是需要注意的是，通过该配置项只能控制在 NewRESTStorage() 方法中定义的版本以及资源，具体可见下面的示例。
  • restOptionGetter就是前文讲过的用来创建 REST Store 以及 etcd store的工厂方法类的实例，其来自于 GenericConfig 中的 RESTOptionsGetter，即上面的 c.GenericConfig.RESTOptionsGetter。

各种资源的RestStorageProvider构建好之后，调用InstallAPIs()，将RESTStorageProvider列表，当做参数传进去，进行安装，先来看下这个安装API的方法：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

# kubernetes/pkg/controlplane/instance.go

func (m *Instance) InstallAPIs(apiResourceConfigSource serverstorage.APIResourceConfigSource, restOptionsGetter generic.RESTOptionsGetter, restStorageProviders ...RESTStorageProvider) error {
    nonLegacy := []*genericapiserver.APIGroupInfo{}
    ......
    for _, restStorageBuilder := range restStorageProviders {
        groupName := restStorageBuilder.GroupName()
        apiGroupInfo, err := restStorageBuilder.NewRESTStorage(apiResourceConfigSource, restOptionsGetter)
        ......
        if len(groupName) == 0 {
            // the legacy group for core APIs is special that it is installed into /api via this special install method.
            if err := m.GenericAPIServer.InstallLegacyAPIGroup(genericapiserver.DefaultLegacyAPIPrefix, &apiGroupInfo); err != nil {
                return fmt.Errorf("error in registering legacy API: %w", err)
            }
        } else {
            // everything else goes to /apis
            nonLegacy = append(nonLegacy, &apiGroupInfo)
        }
    }
    ......
    if err := m.GenericAPIServer.InstallAPIGroups(nonLegacy...); err != nil {
        return fmt.Errorf("error in registering group versions: %v", err)
    }
    return nil
}

可以看到，通过RESTStorageProvider的NewRESTStorage()构造出 APIGroupInfo，然后分别调用了GenericAPIServer的暴露的InstallLegacyAPIGroup()和InstallAPIGroups()方法进行安装注册。下面先来看下这个 APIGroupInfo 是如何构建出来的，以core group为例：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

# kubernetes/pkg/registry/core/rest/storage_core.go

func (c *legacyProvider) NewRESTStorage(restOptionsGetter generic.RESTOptionsGetter) (LegacyRESTStorage, genericapiserver.APIGroupInfo, error) {
    apiGroupInfo, err := c.GenericConfig.NewRESTStorage(apiResourceConfigSource, restOptionsGetter)
    ......
    podStorage, err := podstore.NewStorage(
        restOptionsGetter,
        nodeStorage.KubeletConnectionInfo,
        c.ProxyTransport,
        podDisruptionClient,
    )

    serviceRESTStorage, serviceStatusStorage, serviceRESTProxy, err := servicestore.NewREST(
        restOptionsGetter,
        c.primaryServiceClusterIPAllocator.IPFamily(),
        c.serviceClusterIPAllocators,
        c.serviceNodePortAllocator,
        endpointsStorage,
        podStorage.Pod,
        c.Proxy.Transport)
    ......
    storage := apiGroupInfo.VersionedResourcesStorageMap["v1"]

    if resource := "pods"; apiResourceConfigSource.ResourceEnabled(corev1.SchemeGroupVersion.WithResource(resource)) {
        storage[resource] = podStorage.Pod
        storage[resource+"/attach"] = podStorage.Attach
        storage[resource+"/status"] = podStorage.Status
        storage[resource+"/log"] = podStorage.Log
        storage[resource+"/exec"] = podStorage.Exec
        storage[resource+"/portforward"] = podStorage.PortForward
        storage[resource+"/proxy"] = podStorage.Proxy
        storage[resource+"/binding"] = podStorage.Binding
        if podStorage.Eviction != nil {
            storage[resource+"/eviction"] = podStorage.Eviction
        }
        storage[resource+"/ephemeralcontainers"] = podStorage.EphemeralContainers
    }
    ......
    if resource := "services"; apiResourceConfigSource.ResourceEnabled(corev1.SchemeGroupVersion.WithResource(resource)) {
        storage[resource] = serviceRESTStorage
        storage[resource+"/proxy"] = serviceRESTProxy
        storage[resource+"/status"] = serviceStatusStorage
    }
    ......
    apiGroupInfo.VersionedResourcesStorageMap["v1"] = storage
    return apiGroupInfo, nil
}


# kubernetes/pkg/registry/core/rest/storage_core_generic.go

func (c *GenericConfig) NewRESTStorage(apiResourceConfigSource serverstorage.APIResourceConfigSource, restOptionsGetter generic.RESTOptionsGetter) (genericapiserver.APIGroupInfo, error) {
    apiGroupInfo := genericapiserver.APIGroupInfo{
        PrioritizedVersions:          legacyscheme.Scheme.PrioritizedVersionsForGroup(""),
        VersionedResourcesStorageMap: map[string]map[string]rest.Storage{},
        Scheme:                       legacyscheme.Scheme,
        ParameterCodec:               legacyscheme.ParameterCodec,
        NegotiatedSerializer:         legacyscheme.Codecs,
    }

    secretStorage, err := secretstore.NewREST(restOptionsGetter)
    serviceAccountStorage, err = serviceaccountstore.NewREST(restOptionsGetter, nil, nil, 0, nil, nil, false)
    ......
    if resource := "secrets"; apiResourceConfigSource.ResourceEnabled(corev1.SchemeGroupVersion.WithResource(resource)) {
        storage[resource] = secretStorage
    }

    if resource := "serviceaccounts"; apiResourceConfigSource.ResourceEnabled(corev1.SchemeGroupVersion.WithResource(resource)) {
        storage[resource] = serviceAccountStorage
        if serviceAccountStorage.Token != nil {
            storage[resource+"/token"] = serviceAccountStorage.Token
        }
    }
    ......
    if len(storage) > 0 {
        apiGroupInfo.VersionedResourcesStorageMap["v1"] = storage
    }

    return apiGroupInfo, nil
}

可以看到在这里面首先通过 c.GenericConfig.NewRESTStorage() 方法返回了一个APIGroupInfo，在该方法中，主要是创建在Core Group中通用的资源的REST store，比如 serviceaccounts, secrets等等，然后通过podStore.NewStorage()构造了pod及其subresource的REST store，此外还有 service, nodes等其他资源的REST store，需要注意的是，上例中 podStorage 并不是一个REST store，它只是一个包含了很多REST store的变量而已，它里面的podStorage.Pod, podStorage.Attach才是 REST store，其他资源跟此类似，然后将他们注册到到一个storage map里面，在注册时，还判断了是否要启用这个资源，最终将这个map存储到VersionedResourcesStorageMap对应的版本中。所以storage中存储的是这个Group中v1版本对应的所有的API对象资源的REST store，包括pods, services, nodes等等。

再来看一个named group中的API对象，以apps组中的对象为例：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

# kubernetes/pkg/registry/apps/rest/storage_apps.go

func (p StorageProvider) NewRESTStorage(apiResourceConfigSource serverstorage.APIResourceConfigSource, restOptionsGetter generic.RESTOptionsGetter) (genericapiserver.APIGroupInfo, bool, error) {
    apiGroupInfo := genericapiserver.NewDefaultAPIGroupInfo(apps.GroupName, legacyscheme.Scheme, legacyscheme.ParameterCodec, legacyscheme.Codecs)
    // If you add a version here, be sure to add an entry in `k8s.io/kubernetes/cmd/kube-apiserver/app/aggregator.go with specific priorities.
    // TODO refactor the plumbing to provide the information in the APIGroupInfo

    if storageMap, err := p.v1Storage(apiResourceConfigSource, restOptionsGetter); err != nil {
        return genericapiserver.APIGroupInfo{}, err
    } else if len(storageMap) > 0 {
        apiGroupInfo.VersionedResourcesStorageMap[appsapiv1.SchemeGroupVersion.Version] = storageMap
    }

    return apiGroupInfo, nil
}

func (p StorageProvider) v1Storage(apiResourceConfigSource serverstorage.APIResourceConfigSource, restOptionsGetter generic.RESTOptionsGetter) (map[string]rest.Storage, error) {
    storage := map[string]rest.Storage{}

    // deployments
    if resource := "deployments"; apiResourceConfigSource.ResourceEnabled(appsapiv1.SchemeGroupVersion.WithResource(resource)) {
        deploymentStorage, err := deploymentstore.NewStorage(restOptionsGetter)
        if err != nil {
            return storage, err
        }
        storage[resource] = deploymentStorage.Deployment
        storage[resource+"/status"] = deploymentStorage.Status
        storage[resource+"/scale"] = deploymentStorage.Scale
    }

    // statefulsets
    if resource := "statefulsets"; apiResourceConfigSource.ResourceEnabled(appsapiv1.SchemeGroupVersion.WithResource(resource)) {
        statefulSetStorage, err := statefulsetstore.NewStorage(restOptionsGetter)
        if err != nil {
            return storage, err
        }
        storage[resource] = statefulSetStorage.StatefulSet
        storage[resource+"/status"] = statefulSetStorage.Status
        storage[resource+"/scale"] = statefulSetStorage.Scale
    }
    ......
    return storage, nil
}

这里就可以看到，apps这个组只有一个v1版本可以用，因为它只创建了v1版本的REST store并且注册到storage map中。各种资源的NewStorage()方法的细节，这里就不介绍了，主要是构建对应API对象资源的REST store，跟Core Group类似，也在Kubernetes APIServer Storage 框架解析中介绍REST store的上层应用时有介绍过。

Aggregator

在Aggregator的New()方法中，也有类似上面的逻辑：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

# kube-aggregator/pkg/apiserver/apiserver.go

func (c completedConfig) NewWithDelegate(delegationTarget genericapiserver.DelegationTarget) (*APIAggregator, error) {
    ......
    genericServer, err := c.GenericConfig.New("kube-aggregator", delegationTarget)
    ......
    s := &APIAggregator{
        GenericAPIServer:         genericServer,
        delegateHandler:          delegationTarget.UnprotectedHandler(),
        proxyClientCert:          c.ExtraConfig.ProxyClientCert,
        proxyClientKey:           c.ExtraConfig.ProxyClientKey,
        proxyTransport:           c.ExtraConfig.ProxyTransport,
        proxyHandlers:            map[string]*proxyHandler{},
        handledGroups:            sets.String{},
        lister:                   informerFactory.Apiregistration().V1().APIServices().Lister(),
        APIRegistrationInformers: informerFactory,
        serviceResolver:          c.ExtraConfig.ServiceResolver,
        openAPIConfig:            openAPIConfig,
        egressSelector:           c.GenericConfig.EgressSelector,
    }
    ......
    apiGroupInfo := apiservicerest.NewRESTStorage(c.GenericConfig.MergedResourceConfig, c.GenericConfig.RESTOptionsGetter)

    if err := s.GenericAPIServer.InstallAPIGroup(&apiGroupInfo); err != nil {
        return nil, err
    }
    ......
}

# kube-aggregator/pkg/registry/apiservice/rest/storage_apiservice.go

func NewRESTStorage(apiResourceConfigSource serverstorage.APIResourceConfigSource, restOptionsGetter generic.RESTOptionsGetter, shouldServeBeta bool) genericapiserver.APIGroupInfo {
    apiGroupInfo := genericapiserver.NewDefaultAPIGroupInfo(apiregistration.GroupName, aggregatorscheme.Scheme, metav1.ParameterCodec, aggregatorscheme.Codecs)

    storage := map[string]rest.Storage{}

    if resource := "apiservices"; apiResourceConfigSource.ResourceEnabled(v1.SchemeGroupVersion.WithResource(resource)) {
        apiServiceREST := apiservicestorage.NewREST(aggregatorscheme.Scheme, restOptionsGetter)
        storage[resource] = apiServiceREST
        storage[resource+"/status"] = apiservicestorage.NewStatusREST(aggregatorscheme.Scheme, apiServiceREST)
    }

    if len(storage) > 0 {
        apiGroupInfo.VersionedResourcesStorageMap["v1"] = storage
    }

    return apiGroupInfo
}

Aggregator中，就只有apiservices这一个API对象资源，并且也只有v1这一个版本可以用。

APIExtensions

再来看看APIExtensions的New()方法，也是类似的：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# kube-aggregator/pkg/apiserver/apiserver.go

func (c completedConfig) New(delegationTarget genericapiserver.DelegationTarget) (*CustomResourceDefinitions, error) {
    genericServer, err := c.GenericConfig.New("apiextensions-apiserver", delegationTarget)
    s := &CustomResourceDefinitions{
        GenericAPIServer: genericServer,
    }

    apiResourceConfig := c.GenericConfig.MergedResourceConfig
    apiGroupInfo := genericapiserver.NewDefaultAPIGroupInfo(apiextensions.GroupName, Scheme, metav1.ParameterCodec, Codecs)
    storage := map[string]rest.Storage{}
    // customresourcedefinitions
    if resource := "customresourcedefinitions"; apiResourceConfig.ResourceEnabled(v1.SchemeGroupVersion.WithResource(resource)) {
        customResourceDefinitionStorage, err := customresourcedefinition.NewREST(Scheme, c.GenericConfig.RESTOptionsGetter)
        if err != nil {
            return nil, err
        }
        storage[resource] = customResourceDefinitionStorage
        storage[resource+"/status"] = customresourcedefinition.NewStatusREST(Scheme, customResourceDefinitionStorage)
    }
    if len(storage) > 0 {
        apiGroupInfo.VersionedResourcesStorageMap[v1.SchemeGroupVersion.Version] = storage
    }

    if err := s.GenericAPIServer.InstallAPIGroup(&apiGroupInfo); err != nil {
        return nil, err
    }
}

APIExtensions中也只定义了customresourcedefinitions这一个资源，并且也只有v1这一个版本。

总结

以上，分别介绍了KubeAPIServer, Aggregator和APIExtensions中各自的APIGroupInfo是如何构建的，如何调用到GenericAPIServer中的安装方法进行安装的，可以看到，不同版本的API对象，其实是分别构建了其REST store，即在数据库中独立存储的。下面来盘点下按照上述方式，看Kubernetes API中，都内置了哪些对象，当前Kubernetes最新的版本为1.19.0：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
apiextensions-apiserver
* resources
    * /apis/apiextensions.k8s.io/
        * customresourcedefinations  -> GoRestfulContainer
        * customresourcedefinations/status -> GoRestfulContainer
        * /apis  -> NonGoRestfulMux  -> crdHandler  // Handle()
        * /apis/  -> NonGoRestfulMux  -> crdHandler  // HandlePrefix()，CRD定义的自定义资源的CRUD操作都在这个Handler中操作

apiserver
* resources
    * /api/v1  -> GoRestfulContainer
        * pods
        * pods/attach
        * pods/status
        * pods/log
        * pods/exec
        * pods/portforward
        * pods/proxy
        * pods/binding
        * pods/eviction
        * pods/ephemeralcontainers
        * bindings
        * podTemplates
        * replicationControllers
        * replicationControllers/status
        * replicationControllers/scale
        * services
        * services/proxy
        * services/status
        * endpoints
        * nodes
        * nodes/status
        * nodes/proxy
        * events
        * limitRanges
        * resourceQuotas
        * resourceQuotas/status
        * namespaces
        * namespaces/status
        * namespaces/finalize
        * secrets
        * serviceAccounts
        * serviceAccounts/token
        * persistentVolumes
        * persistentVolumes/status
        * persistentVolumeClaims
        * persistentVolumeClaims/status
        * configMaps
        * componentStatuses
    * /apis  -> GoRestfulContainer
        * authentication.k8s.io
            * tokenreviews
        * authorization.k8s.io
            * subjectaccessreviews
            * selfsubjectaccessreviews
            * localsubjectaccessreviews
            * selfsubjectrulesreviews
        * autoscaling
            * horizontalpodautoscalers
            * horizontalpodautoscalers/status
        * batch
            * v1
                * jobs
                * jobs/status
            * v1beta1
                * cronjobs
                * cronjobs/status
            * v2alpha1
                * cronjobs
                * cronjobs/status
        * certificates.k8s.io
            * certificatesigningrequests
            * certificatesigningrequests/status
            * certificatesigningrequests/approval
        * coordination.k8s.io
            * leases
        * discovery.k8s.io
            * endpointslices
        * extensions
            * v1beta1
                * ingresses
                * ingresses/status
        * networking.k8s.io
            * v1
                * networkpolicies
            * v1beta1
                * ingresses
                * ingresses/status
                * ingressclasses
        * node.k8s.io
            * v1alpha1
                * runtimeclasses
            * v1beta1
                * runtimeclasses
        * policy
            * v1beta1
                * poddisruptionbudgets
                * poddisruptionbudgets/status
                * podsecuritypolicies
        * rbac.authorization.k8s.io
            * roles
            * rolebindings
            * clusterroles
            * clusterrolebindings
        * scheduling.k8s.io
            * priorityclasses
        * settings.k8s.io
            * podpresets
        * storage.k8s.io
            * v1alpha1
                * volumeattachments
            * v1beta1
                * storageclasses
                * volumeattachments
                * csinodes
                * csidrivers
            * v1
                * storageclasses
                * volumeattachments
                * volumeattachments/status
                * csinodes
                * csidrivers
        * flowcontrol.apiserver.k8s.io
            * flowschemas
            * flowschemas/status
            * prioritylevelconfigurations
            * prioritylevelconfigurations/status
        * apps
            * deployments
            * deployments/status
            * deployments/scale
            * statefulsets
            * statefulsets/status
            * statefulsets/scale
            * daemonsets
            * daemonsets/status
            * replicasets
            * replicasets/status
            * replicasets/scale
            * controllerrevisions
        * admissionregistration.k8s.io
            * validatingwebhookconfigurations
            * mutatingwebhookconfigurations
        * events.k8s.io
            * events

aggregator
* resources
    * /apis  -> GoRestfulContainer
        * apiregistration.k8s.io
            * apiservices
            * apiservices/status
    * /apis  -> apisHandler  -> NonGoRestfulMux
    * /apis/  -> apisHandler  -> NonGoRestfulMux
    * "/apis/" + apiService.Spec.Group + "/" + apiService.Spec.Version -> proxyHandler -> NonGoRestfulMux
        * 在该proxyHandler中，最终将请求proxy给extension-apiserver
        * 在apiservice-registration-controller poststarthook中通过AddAPIService在添加APIService时，注册进proxyHandler中
    * "/apis/" + apiService.Spec.Group -> groupDiscoveryHandler -> NonGoRestfulMux

可以看到当前版本的Kubernetes API已经非常丰富了，Group就有19个之多，以后内置的API对象肯定还会不断添加，再结合CRD和Aggregator进行扩展，这云原生的头把交椅真不是盖的。

Kubernetes APIServer API Resource Installation

https://hackerain.me/2020/10/06/kubernetes/kube-apiserver-api-resource-installation.html

作者

hackerain

发布于

2020-10-06

更新于

2023-10-28

许可协议

#
关注我

目录

最新文章

归档

标签

ceph2
elasticsearch1
kubernetes20
openstack19
operations1
python1
umi1
×