GCP Cloud Setup

To complete the "Connect to Your Cloud Account" step in the deX platform, you'll need to configure a few resources in your GCP console. Once this is complete, you’ll receive a Service Account JSON key to use within dex.

Create a GCP Project for dex

1. Open the Google Cloud Console.

2. Click New Project.

3. Fill in the required fields:

   • Name your project something like your-org-lakehouse

   • Choose a Location with your SRE or infra team for optimal cost/performance

4. Click Create

Tip: In production, you may want separate projects for dev and prod. For this quickstart, one is enough.

Creating a New Project on Google Cloud Console

Create Cloud Credentials

dex needs specific GCP permissions to operate. We’ve made this easy with a setup script.

For reference, these are the permissions dex needs (generated by the setup script). You can read the full list of permissions and what they do here.

storage.managedFolders.delete
storage.managedFolders.get
storage.managedFolders.list
storage.multipartUploads.abort
storage.multipartUploads.create
storage.multipartUploads.list
storage.multipartUploads.listParts
storage.objects.create
storage.objects.delete
storage.objects.get
storage.objects.list
storage.objects.restore
storage.objects.update
bigquery.datasets.create
bigquery.datasets.get
bigquery.datasets.getIamPolicy
bigquery.jobs.create
bigquery.models.getMetadata
bigquery.models.list
bigquery.routines.get
bigquery.routines.list
bigquery.tables.create
bigquery.tables.get
bigquery.tables.getData
bigquery.tables.getIamPolicy
bigquery.tables.list
bigquery.tables.update
bigquery.tables.updateData
dataplex.projects.search
resourcemanager.projects.get

Set Up GCP Credentials with Cloud Shell

To configure access between dex and BigQuery, follow these steps using Google Cloud Shell:

1. Open the GCP Console

2. In the top bar, click on the project selector and choose the project you created or reserved for dex.

3. Launch Cloud Shell Editor - Use the search bar at the top of the page to search for Cloud Shell Editor and click to open it.

4. Inside the Cloud Shell Editor, click the "Open Terminal" button near the top of the screen.

5. Paste the following command into the terminal to download the setup script:

wget https://raw.githubusercontent.com/dexlabsio/terraform-modules/refs/heads/main/gcp/cloud-shell/bigquery-service-account-setup.py

6. Set your GCP project ID in the shell environment

export GOOGLE_CLOUD_PROJECT=$(gcloud config get-value project)

7. Execute the setup script

python3 bigquery-service-account-setup.py

8. Download your Service Account Key

After running the script, you’ll receive a Service Account JSON key on the screen

Download and store this key securely — you’ll need it when connecting dex to BigQuery.