Comment on page

Analyzing GKE costs with GKE Usage Metering

Prerequisites

Enabling GKE usage metering

To enable the GKE usage metering, you must first [create a BigQuery dataset](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-usage-metering#create-dataset. The dataset should be created for either a single cluster, multiple clusters in the project, or the entire project. For more information about choosing a mapping between datasets and clusters, checkout the documentation on choosing one or more BigQuery datasets.
Next enable the GKE usage metering when creating a new cluster or by modifying an existing cluster.
Optionally, create a Google Data Studio dashboard to visualize the resource usage of your clusters.
If you have created a dashboard using the template provided by Google with the initial Beta, you must first create a new dashboard from the new template in order to visualize actual resource consumption.

Creating the BigQuery dataset

To use the GKE usage metering for clusters in your Google Cloud project, you must first create the BigQuery dataset, and then configure clusters to use it. You can use a single BigQuery dataset to store information about resource usage for multiple clusters in the same project.
Visit creating datasets to know more details. Set the Default table expiration for the dataset to Never so that the table doesn't expire. However, in the second GKE usage metering Beta, if a table expires, it is recreated automatically (as an empty table).
This feature writes to the BigQuery dataset that you provide with the Google service account (service-PROJECT_ID@container-engine-robot.iam.gserviceaccount.com) and with the Kubernetes Engine Service Agent role.
If you delete a BigQuery dataset or table that a cluster is using to log the GKE usage metering data, Cloud Logging shows transient warnings such as Failed to upload a record to BigQuery. To resolve the warning, re-create the dataset or configure the cluster to use a different dataset. Your historical data will be lost.

Enabling GKE usage metering for a cluster

You can enable the GKE usage metering on a new or existing cluster, using either the gcloud command or the Cloud Console.
Enabling GKE usage metering also enables resource consumption metering by default. To selectively disable resource consumption metering while continuing to track resource requests, see the specific instructions for enabling GKE usage metering using the gcloud command, in this topic.
Network egress metering is disabled by default. To enable it, check out the caveats and instructions in Optional: Enabling network egress metering in this topic.
1. Go to the Google Kubernetes Engine page in Cloud Console.
2. Next to the cluster you want to modify, click Actions, then click Edit.
3. Under Features, click Edit next to GKE usage metering.
4. Select Enable GKE usage metering.
5. Enter the name of the BigQuery dataset.
6. Optional: select Enable network egress metering after reviewing the caveats and instructions in Optional: Enabling network egress metering.
7. Click Save Changes.

Integrating with Economize

1. Go to the Kuberenetes Clusters page on Google Cloud.
2. Click on the cluster that you have enabled GKE Usage metering.
3. Under the features section, you can find GKE Usage Metering along with the Dataset name. Clicking on it will take you to the respective dataset.
4. On the left pane, you can expand on your dataset name to find the table called gke_cluster_resource_usage.
5. Click on the Share -> Add Principal.
6. Enter economize's service account as principal, select BigQuery Data Edior on Role and then Click on Save.
7. Click on the details tab to find your table id and copy that, as seen in the screenshot below.
6. Go to Organisations tab on settings page. Click on the pen icon near to your desired project.
7. Paste the table id in the GKE Usage metering table field and click on test to test the connection.