Thanos packaged by Bitnami¶

Thanos is a highly available metrics system that can be added on top of existing Prometheus deployments, providing a global query view across all Prometheus installations.

Overview of Thanos

Trademarks: This software listing is packaged by Bitnami. The respective trademarks mentioned in the offering are owned by the respective companies, and use of them does not imply any affiliation or endorsement.

TL;DR¶

使用加速地址添加仓库:

helm repo add bitnami "https://helm-charts.itboon.top/bitnami" --force-update
helm update bitnami

helm install my-release bitnami/thanos

Introduction¶

This chart bootstraps a Thanos deployment on a Kubernetes cluster using the Helm package manager.

Bitnami charts can be used with Kubeapps for deployment and management of Helm Charts in clusters.

Looking to use Thanos in production? Try VMware Application Catalog, the enterprise edition of Bitnami Application Catalog.

Prerequisites¶

• Kubernetes 1.19+
• Helm 3.2.0+
• PV provisioner support in the underlying infrastructure

Installing the Chart¶

To install the chart with the release name my-release:

helm install my-release bitnami/thanos

These commands deploy Thanos on the Kubernetes cluster with the default configuration. The configuration section lists the parameters that can be configured during installation.

Tip: List all releases using helm list

Uninstalling the Chart¶

To uninstall/delete the my-release chart:

helm uninstall my-release

Architecture¶

This charts allows you install several Thanos components, so you deploy an architecture as the one below:

                       +--------------+                  +--------------+      +--------------+
                       | Thanos       |----------------> | Thanos Store |      | Thanos       |
                       | Query        |           |      | Gateway      |      | Compactor    |
                       +--------------+           |      +--------------+      +--------------+
                   push                           |             |                     |
+--------------+   alerts   +--------------+      |             | storages            | Downsample &
| Alertmanager | <----------| Thanos       | <----|             | query metrics       | compact blocks
| (*)          |            | Ruler        |      |             |                     |
+--------------+            +--------------+      |             \/                    |
      ^                            |              |      +----------------+           |
      | push alerts                +--------------|----> | MinIO&reg; (*) | <---------+
      |                                           |      |                |
+------------------------------+                  |      +----------------+
|+------------+  +------------+|                  |             ^
|| Prometheus |->| Thanos     || <----------------+             |
|| (*)        |<-| Sidecar (*)||    query                       | inspect
|+------------+  +------------+|    metrics                     | blocks
+------------------------------+                                |
                                                         +--------------+
                                                         | Thanos       |
                                                         | Bucket Web   |
                                                         +--------------+

Note: Components marked with (*) are provided by subchart(s) (such as the Bitnami MinIO® chart) or external charts (such as the Bitnami kube-prometheus chart).

Check the section Integrate Thanos with Prometheus and Alertmanager for detailed instructions to deploy this architecture.

Parameters¶

Global parameters¶

Common parameters¶

Thanos common parameters¶

Thanos Query parameters¶

Thanos Query Frontend parameters¶

Thanos Bucket Web parameters¶

Thanos Compactor parameters¶

Thanos Store Gateway parameters¶

Thanos Ruler parameters¶

Thanos Receive parameters¶

Thanos Receive Distributor parameters¶

Metrics parameters¶

Volume Permissions parameters¶

MinIO® chart parameters¶

NetWorkPolicy parameters¶

Specify each parameter using the --set key=value[,key=value] argument to helm install. For example,

helm install my-release --set query.replicaCount=2 bitnami/thanos

The above command install Thanos chart with 2 Thanos Query replicas.

Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example,

helm install my-release -f values.yaml bitnami/thanos

Tip: You can use the default values.yaml

Configuration and installation details¶

Rolling VS Immutable tags¶

It is strongly recommended to use immutable tags in a production environment. This ensures your deployment does not change automatically if the same tag is updated with a different image.

Bitnami will release a new chart updating its containers if a new version of the main container, significant changes, or critical vulnerabilities exist.

Adding extra flags¶

In case you want to add extra flags to any Thanos component, you can use XXX.extraFlags parameter(s), where XXX is placeholder you need to replace with the actual component(s). For instance, to add extra flags to Thanos Store Gateway, use:

storegateway:
  extraFlags:
    - --sync-block-duration=3m
    - --chunk-pool-size=2GB

This also works for multi-line flags. This can be useful when you want to configure caching for a particular component without using a configMap. For example, to configure the query-range response cache of the Thanos Query Frontend, use:

queryFrontend:
  extraFlags:
    - |
      --query-range.response-cache-config=
      type: MEMCACHED
      config:
        addresses:
          - <MEMCACHED_SERVER>:11211
        timeout: 500ms
        max_idle_connections: 100
        max_async_concurrency: 10
        max_async_buffer_size: 10000
        max_get_multi_concurrency: 100
        max_get_multi_batch_size: 0
        dns_provider_update_interval: 10s
        expiration: 24h

Using custom Objstore configuration¶

This helm chart supports using custom Objstore configuration.

You can specify the Objstore configuration using the objstoreConfig parameter.

In addition, you can also set an external Secret with the configuration file. This is done by setting the existingObjstoreSecret parameter. Note that this will override the previous option. If needed you can also provide a custom Secret Key with existingObjstoreSecretItems, please be aware that the Path of your Secret should be objstore.yml.

Using custom Query Service Discovery configuration¶

This helm chart supports using custom Service Discovery configuration for Query.

You can specify the Service Discovery configuration using the query.sdConfig parameter.

In addition, you can also set an external ConfigMap with the Service Discovery configuration file. This is done by setting the query.existingSDConfigmap parameter. Note that this will override the previous option.

Using custom Ruler configuration¶

This helm chart supports using custom Ruler configuration.

You can specify the Ruler configuration using the ruler.config parameter.

In addition, you can also set an external ConfigMap with the configuration file. This is done by setting the ruler.existingConfigmap parameter. Note that this will override the previous option.

Running Thanos with HTTPS and basic authentication¶

This helm charts supports using HTTPS and basic authentication. The underlying feature is experimental and might change in the future, so are the associated settings in the chart. For more information, please refer to Thanos documentation.

This feature can be enabled by using the following values:

• https.enabled=true. Enabling HTTPS requires the user to provide the TLS certificate and Key for Thanos, which can be done using one of the following options:

• Provide a secret using https.existingSecret. The secret must contain the keys tls.crt or tls.key (key names can be renamed using the values https.keyFilename and https.certFilename).

• Provide the certificate and key in your values.yaml under the values https.cert and https.key.

• Use https.autoGenerated=true, using this value Helm will generate a self-signed key pair during the chart initialization. Not recommended for production environments.

• auth.basicAuthUsers.*. An dictionary of key / values, where the keys corresponds to the users that will have access to Thanos and the values are the plaintext passwords. Passwords will be later encrypted with bcrypt.

• Alternatively, provide your own Thanos http config file using the value httpConfig or existingHttpConfigSecret. This may cause any settings under https.* or auth.* to be ignored, except for the settings related to the TLS certificates. When providing a configuration file using these parameters, the chart Probes will fail to initialize unless one of the following fixes are applied:
• Set https.enabled or auth.basicAuthUsers with at least one user, matching the configuration file you provided. That way Probes will be configured with HTTPS and/or basic authentication accordingly.
• Configure your own Probes using <component>.customLivenessProbe, <component>.customReadinessProbe and <component>.customStartupProbe.
• Not recommended. Disable the Probes.

Store time partitions¶

Thanos store supports partion based on time.

Setting time partitions will create N number of store statefulsets based on the number of items in the timePartitioning list. Each item must contain the min and max time for querying in the supported format (find more details at Thanos documentation).

Note: leaving the timePartitioning list empty ([]) will create a single store for all data.

For instance, to use 3 stores you can use a values.yaml like the one below:

timePartitioning:
  # One store for data older than 6 weeks
  - min: ""
    max: -6w
  # One store for data newer than 6 weeks and older than 2 weeks
  - min: -6w
    max: -2w
  # One store for data newer than 2 weeks
  - min: -2w
    max: ""

You can also specify different resources and limits configurations for each storegateway statefulset. This is done by adding a resources.requests and resources.limits to each item you wish to change, as shown below:

timePartitioning:
  # One store for data older than 6 weeks
  - min: ""
    max: -6w
  # One store for data newer than 6 weeks and older than 2 weeks
  - min: -6w
    max: -2w
    resources: #optional resources declaration for partition
      requests:
        cpu: 10m
        memory: 100Mi
      limits:
        cpu: 20m
        memory: 100Mi
  # One store for data newer than 2 weeks
  - min: -2w
    max: ""

Integrate Thanos with Prometheus and Alertmanager¶

You can integrate Thanos with Prometheus & Alertmanager using this chart and the Bitnami kube-prometheus chart following the steps below:

Note: in this example we will use MinIO® (subchart) as the Objstore. Every component will be deployed in the "monitoring" namespace.

• Create a values.yaml like the one below:

objstoreConfig: |-
  type: s3
  config:
    bucket: thanos
    endpoint: {{ include "thanos.minio.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local:9000
    access_key: minio
    secret_key: minio123
    insecure: true
query:
  dnsDiscovery:
    sidecarsService: kube-prometheus-prometheus-thanos
    sidecarsNamespace: monitoring
bucketweb:
  enabled: true
compactor:
  enabled: true
storegateway:
  enabled: true
ruler:
  enabled: true
  alertmanagers:
    - http://kube-prometheus-alertmanager.monitoring.svc.cluster.local:9093
  config: |-
    groups:
      - name: "metamonitoring"
        rules:
          - alert: "PrometheusDown"
            expr: absent(up{prometheus="monitoring/kube-prometheus"})
metrics:
  enabled: true
  serviceMonitor:
    enabled: true
minio:
  enabled: true
  auth:
    rootPassword: minio123
    rootUser: minio
  monitoringBuckets: thanos
  accessKey:
    password: minio
  secretKey:
    password: minio123

• Install Prometheus Operator and Thanos charts:

For Helm 3:

$ kubectl create namespace monitoring
helm install kube-prometheus \
    --set prometheus.thanos.create=true \
    --namespace monitoring \
    bitnami/kube-prometheus
helm install thanos \
    --values values.yaml \
    --namespace monitoring \
    bitnami/thanos

That's all! Now you have Thanos fully integrated with Prometheus and Alertmanager.

Persistence¶

The data is persisted by default using PVC(s) on Thanos components. You can disable the persistence setting the XXX.persistence.enabled parameter(s) to false. A default StorageClass is needed in the Kubernetes cluster to dynamically provision the volumes. Specify another StorageClass in the XXX.persistence.storageClass parameter(s) or set XXX.persistence.existingClaim if you have already existing persistent volumes to use.

Note: you need to substitute the XXX placeholders above with the actual component(s) you want to configure.

Adjust permissions of persistent volume mountpoint¶

As the images run as non-root by default, it is necessary to adjust the ownership of the persistent volumes so that the containers can write data into it.

By default, the chart is configured to use Kubernetes Security Context to automatically change the ownership of the volumes. However, this feature does not work in all Kubernetes distributions. As an alternative, this chart supports using an initContainer to change the ownership of the volumes before mounting it in the final destination.

You can enable this initContainer by setting volumePermissions.enabled to true.

Deploy extra resources¶

There are cases where you may want to deploy extra objects, such a ConfigMap containing your app's configuration or some extra deployment with a micro service used by your app. For covering this case, the chart allows adding the full specification of other objects using the extraDeploy parameter.

Setting Pod's affinity¶

This chart allows you to set your custom affinity using the XXX.affinity parameter(s). Find more information about Pod's affinity in the kubernetes documentation.

As an alternative, you can use of the preset configurations for pod affinity, pod anti-affinity, and node affinity available at the bitnami/common chart. To do so, set the XXX.podAffinityPreset, XXX.podAntiAffinityPreset, or XXX.nodeAffinityPreset parameters.

Troubleshooting¶

Find more information about how to deal with common errors related to Bitnami's Helm charts in this troubleshooting guide.

Upgrading¶

To 12.0.0¶

This major updates the MinIO® subchart to its newest major, 12.0.0. This subchart's major doesn't include any changes affecting its use as a subchart for Thanos, so no major issues are expected during the upgrade.

To 10.0.0¶

This version deprecates the usage of MINIO_ACCESS_KEY and MINIO_SECRET_KEY environment variables in MINIO® container in favor of MINIO_ROOT_USER and MINIO_ROOT_PASSWORD.

If you were already using the new variables, no issues are expected during upgrade.

To 9.0.0¶

This major updates the MinIO® subchart to its newest major, 10.0.0. This subchart's major doesn't include any changes affecting its use as a subchart for Thanos, only needing the standard upgrade process from chart's version 8.X.

To 8.0.0¶

The chart was changed to adapt to the common Bitnami chart standards. Now it includes common elements such as sidecar and init container support, custom commands, custom liveness/readiness probes, extra environment variables support, extra pod annotations and labels, among others. In addition, other remarkable changes were to harmonize the values structure:

• The receive.distributor.* parameters have been renamed to receiveDistributor.*, and the associated manifests have been moved into its own folder. This way, Thanos Receive Distributor is treated as any other component.
• The parameter to configure TLS in the GRPC server have been restructured so they follow the XXX.grpc.server.tls and XXX.grpc.client.tls format. Previous to this change, we had different structures depending on the component.

To 7.0.0¶

This major updates the MinIO® subchart to its newest major, 9.0.0, which updates authentication parameters. Check MinIO® Upgrading Notes for more information. This major release renames several values in this chart and adds missing features, in order to be inline with the rest of assets in the Bitnami charts repositor. Some of the affected values are:

• XXX.extraEnv parameters were renamed to XXX.extraEnvVars.
• XXX.extraContainers parameters were renamed to XXX.sidecars.
• XXX.service.PROTOCOL.port parameters were renamed to XXX.service.ports.PROTOCOL (e.g. query.service.http.port is now query.service.ports.http).
• XXX.service.PROTOCOL.nodePortparameters were renamed to XXX.service.nodePorts.PROTOCOL (e.g. query.service.http.nodePort is now query.service.nodePorts.http).
• XXX.ingress.certManager parameters were deprecated, use XXX.ingress.annotations to set the required annotations for CertManager instead.

To 6.0.0¶

This major updates the MinIO® subchart to its newest major, 8.0.0, which now has two separated services for MinIO® Console and MinIO® API. Check MinIO® Upgrading Notes for more information.

To 5.4.0¶

This version introduces support for the receiver dual-mode implementation for Thanos v0.22+

To 5.3.0¶

This version introduces hash and time partitioning for the store gateway.

To 5.0.0¶

This major update changes the securityContext interface in the values.yaml file.

Please note if you have changes in the securityContext fields those need to be migrated to podSecurityContext.

# ...
- securityContext:
+ podSecurityContext:
# ...

Other than that a new securityContext interface for containers got introduced containerSecurityContext. It's default is enabled so if you do not need it you need to opt out of it.

# ...
+ containerSecurityContext
+   enabled: true  # opt out by enabled: false
+   capabilities:
+     drop:
+     - ALL
+   runAsNonRoot: true
+   allowPrivilegeEscalation: false
+   readOnlyRootFilesystem: false
# ...

To 4.0.0¶

This major updates the MinIO subchart to its newest major, 7.0.0, which removes previous configuration of securityContext and moves to podSecurityContext and containerSecurityContext.

To 3.3.0¶

This version introduces bitnami/common, a library chart as a dependency. More documentation about this new utility could be found here. Please, make sure that you have updated the chart dependencies before executing any upgrade.

To 3.1.0¶

The querier component and its settings have been renamed to query. Configuration of the query component by using keys under querier in your values.yaml will continue to work. Support for keys under querier will be dropped in a future release.

querier.enabled                               -> query.enabled
querier.logLevel                              -> query.logLevel
querier.replicaLabel                          -> query.replicaLabel
querier.dnsDiscovery.enabled                  -> query.dnsDiscovery.enabled
querier.dnsDiscovery.sidecarsService          -> query.dnsDiscovery.sidecarsService
querier.dnsDiscovery.sidecarsNamespace        -> query.dnsDiscovery.sidecarsNamespace
querier.stores                                -> query.stores
querier.sdConfig                              -> query.sdConfig
querier.existingSDConfigmap                   -> query.existingSDConfigmap
querier.extraFlags                            -> query.extraFlags
querier.replicaCount                          -> query.replicaCount
querier.strategyType                          -> query.strategyType
querier.affinity                              -> query.affinity
querier.nodeSelector                          -> query.nodeSelector
querier.tolerations                           -> query.tolerations
querier.podLabels                             -> query.podLabels
querier.priorityClassName                     -> query.priorityClassName
querier.securityContext.enabled               -> query.securityContext.enabled
querier.securityContext.fsGroup               -> query.securityContext.fsGroup
querier.securityContext.runAsUser             -> query.securityContext.runAsUser
querier.resources.limits                      -> query.resources.limits
querier.resources.requests                    -> query.resources.requests
querier.podAnnotations                        -> query.podAnnotations
querier.livenessProbe                         -> query.livenessProbe
querier.readinessProbe                        -> query.readinessProbe
querier.grpcTLS.server.secure                 -> query.grpcTLS.server.secure
querier.grpcTLS.server.cert                   -> query.grpcTLS.server.cert
querier.grpcTLS.server.key                    -> query.grpcTLS.server.key
querier.grpcTLS.server.ca                     -> query.grpcTLS.server.ca
querier.grpcTLS.client.secure                 -> query.grpcTLS.client.secure
querier.grpcTLS.client.cert                   -> query.grpcTLS.client.cert
querier.grpcTLS.client.key                    -> query.grpcTLS.client.key
querier.grpcTLS.client.ca                     -> query.grpcTLS.client.ca
querier.grpcTLS.client.servername             -> query.grpcTLS.client.servername
querier.service.type                          -> query.service.type
querier.service.clusterIP                     -> query.service.clusterIP
querier.service.http.port                     -> query.service.http.port
querier.service.http.nodePort                 -> query.service.http.nodePort
querier.service.grpc.port                     -> query.service.grpc.port
querier.service.grpc.nodePort                 -> query.service.grpc.nodePort
querier.service.loadBalancerIP                -> query.service.loadBalancerIP
querier.service.loadBalancerSourceRanges      -> query.service.loadBalancerSourceRanges
querier.service.annotations                   -> query.service.annotations
querier.service.labelSelectorsOverride        -> query.service.labelSelectorsOverride
querier.serviceAccount.annotations            -> query.serviceAccount.annotations
querier.rbac.create                           -> query.rbac.create
querier.pspEnabled                            -> query.pspEnabled
querier.autoscaling.enabled                   -> query.autoscaling.enabled
querier.autoscaling.minReplicas               -> query.autoscaling.minReplicas
querier.autoscaling.maxReplicas               -> query.autoscaling.maxReplicas
querier.autoscaling.targetCPU                 -> query.autoscaling.targetCPU
querier.autoscaling.targetMemory              -> query.autoscaling.targetMemory
querier.pdb.create                            -> query.pdb.create
querier.pdb.minAvailable                      -> query.pdb.minAvailable
querier.pdb.maxUnavailable                    -> query.pdb.maxUnavailable
querier.ingress.enabled                       -> query.ingress.enabled
querier.ingress.certManager                   -> query.ingress.certManager
querier.ingress.hostname                      -> query.ingress.hostname
querier.ingress.annotations                   -> query.ingress.annotations
querier.ingress.tls                           -> query.ingress.tls
querier.ingress.extraHosts[0].name            -> query.ingress.extraHosts[0].name
querier.ingress.extraHosts[0].path            -> query.ingress.extraHosts[0].path
querier.ingress.extraTls[0].hosts[0]          -> query.ingress.extraTls[0].hosts[0]
querier.ingress.extraTls[0].secretName        -> query.ingress.extraTls[0].secretName
querier.ingress.secrets[0].name               -> query.ingress.secrets[0].name
querier.ingress.secrets[0].certificate        -> query.ingress.secrets[0].certificate
querier.ingress.secrets[0].key                -> query.ingress.secrets[0].key
querier.ingress.grpc.enabled                  -> query.ingress.grpc.enabled
querier.ingress.grpc.certManager              -> query.ingress.grpc.certManager
querier.ingress.grpc.hostname                 -> query.ingress.grpc.hostname
querier.ingress.grpc.annotations              -> query.ingress.grpc.annotations
querier.ingress.grpc.extraHosts[0].name       -> query.ingress.grpc.extraHosts[0].name
querier.ingress.grpc.extraHosts[0].path       -> query.ingress.grpc.extraHosts[0].path
querier.ingress.grpc.extraTls[0].hosts[0]     -> query.ingress.grpc.extraTls[0].hosts[0]
querier.ingress.grpc.extraTls[0].secretName   -> query.ingress.grpc.extraTls[0].secretName
querier.ingress.grpc.secrets[0].name          -> query.ingress.grpc.secrets[0].name
querier.ingress.grpc.secrets[0].certificate   -> query.ingress.grpc.secrets[0].certificate
querier.ingress.grpc.secrets[0].key           -> query.ingress.grpc.secrets[0].key

To 3.0.0¶

On November 13, 2020, Helm v2 support was formally finished, this major version is the result of the required changes applied to the Helm Chart to be able to incorporate the different features added in Helm v3 and to be consistent with the Helm project itself regarding the Helm v2 EOL.

Learn more about this change and related upgrade considerations.

To 2.4.0¶

The Ingress API object name for Querier changes from {{ include "common.names.fullname" . }} to {{ include "common.names.fullname" . }}-querier.

NOTE: Which in most cases (depending on any set values in fullnameOverride or nameOverride) resolves to the used Helm release name (.Release.Name).

To 2.0.0¶

The format of the chart's extraFlags option has been updated to be an array (instead of an object), to support passing multiple flags with the same name to Thanos.

Now you need to specify the flags in the following way in your values file (where component is one of querier/bucketweb/compactor/storegateway/ruler):

component:
  ...
  extraFlags
    - --sync-block-duration=3m
    - --chunk-pool-size=2GB

To specify the values via CLI::

--set 'component.extraFlags[0]=--sync-block-duration=3m' --set 'ruler.extraFlags[1]=--chunk-pool-size=2GB'

To 1.0.0¶

If you are upgrading from a <1.0.0 release you need to move your Querier Ingress information to the new values settings:

ingress.enabled -> querier.ingress.enabled
ingress.certManager -> querier.ingress.certManager
ingress.hostname -> querier.ingress.hostname
ingress.annotations -> querier.ingress.annotations
ingress.extraHosts[0].name -> querier.ingress.extraHosts[0].name
ingress.extraHosts[0].path -> querier.ingress.extraHosts[0].path
ingress.extraHosts[0].hosts[0] -> querier.ingress.extraHosts[0].hosts[0]
ingress.extraHosts[0].secretName -> querier.ingress.extraHosts[0].secretName
ingress.secrets[0].name -> querier.ingress.secrets[0].name
ingress.secrets[0].certificate -> querier.ingress.secrets[0].certificate
ingress.secrets[0].key -> querier.ingress.secrets[0].key

License¶

Copyright © 2023 VMware, Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.