Google Analytics and BigQuery: setting up your project
With the onset of Universal Analytics sunsetting, and the fab Google Analytics 4 dataset, there has been a considerable uplift in users needing to set up BigQuery Projects to Stream their BigQuery data. In this post, I will walk you through step by step of setting up a Google Cloud Platform and talk about some of the tricks and advice from my experience of working with Google Cloud Platform.
What is the Google Cloud Platform (GCP)?
In GCP’s words “Google Cloud projects form the basis for creating, enabling, and using all Google Cloud services including managing APIs, enabling billing, adding and removing collaborators, and managing permissions for Google Cloud resources.” In plain speak, it’s a group of tools which support an end to end management of data warehouses, apps, web development and much more.
In this post, I am going to talk you through how to set up a basic project with a focus on BigQuery tools.
What is it used for?
The use cases are pretty extensive however these are some of the examples of the type of work I used GCP for:
- Automation of pipelines to stream data into BigQuery
- Linking Data Studio and other visualisation tools to your data
- Cloud Storage and centralisation of information
- APIs and access management
- Data Warehousing and Data Lakes
- Machine learning
- App development
Table of Contents
– Setting up your Google Cloud Platform Account
– Setting up your first project
– Billing account set up
– Identify Access Management (IAMs)
– Connecting useful APIs
Setting up your Google Cloud Platform Account
Before you sign up, make sure you have a payment method available to you (if using a company card, ensure the person who verifies this account is aware and on hand to provide the verification code. You will initially be granted a $300 free allowance for 90 days.
If a billing account has already been set up, get your colleague to invite you to a project and you can then sign up to GCP via that link without inputting your billing information.
Firstly head to https://cloud.google.com/, if you have a Gmail account, then simply sign in, and for those who have another email type such as outlook, sign in with use my current email address instead. You will then be taken through steps to create an account.
Setting up your first project
In this section we will:
- Sign up for a GCP account
- Set up your first Project
Now you’ve signed up you’ll be taken to the GCP interface and see My First Project. I personally recommend deleting this project after you’ve set up your project.
Before you set up your project, I highly suggest considering the following:
- Define what this project will be for and name it accordingly, but not be too specific. SO for example “great-org-marketing-warehouse” rather than “great-org-ga4-warehouse”. This allows for additions in data and changes in the tech stack.
- Selecting what default region your data will be stored in and the name of your project, I suggest the location of your head office but review this if you are working with data from the US or other country which has specific information-sharing laws
- Who will be owning and managing the project and how it will be used
I will be doing a step by step guide on creating and managing a data warehouse in my next post which delves into this further.
Steps
Step 1: Click on the drop-down for the projects, and select “New Project”
Step 2: If you have been added to an organisation you will have the option to select this here, it’s super important you do this now, as you will save a lot of effort later on. I’ll be writing soon about how to set up your organisations and migrate projects. For now, select “New Project”.
Step 3: Add a name to your project, this will need to be lower case and words separated by dashes. You should also set your project id, which needs to be globally unique. I personally suggest ensuring this is not guessable externally but easy for your colleagues using the project. If you do not choose to edit the project ID you will see a randomly generated id such as “braided-carport-350708”. Click Create.
Next steps
Now you have a project set up, we will need to add some functionality to your project
- Billing account set up
- Identify Access Management
Billing account set up
If you have not already got a billing account set up in your company, you should now set up one. Navigate to the Billing account dashboard via the “hamburger” menu. If you have a linked billing account you can select this, likewise, you can select manage billing account to see available billing accounts or create a billing account.
If creating a new billing account, be sure to add a relevant name to the billing account, for example, a departmental name.
The billing account sits outside of the GCP projects, and you can have multiple projects, usually up to five, attached to a billing account. You are able to move projects between billing accounts and edit and rename these in the future.
I strongly suggest having at least two principles for a billing account and you can do this by navigating to the GCP billing dashboard> selecting account management and then adding principle.
Connect your project to a billing account
To be able to add a project to a billing account, you will need to have the relevant permissions if you are not a principal to the billing account, alternatively, you can grant access to your project to your billing account administrator. The permissions required in IAMs are Project Billing Manager.
With the permissions set up, now navigate to BigQuery and you will see the following banner, select upgrade. Now select the billing account you wish to connect the project to and set the account.
Identity Access Management (IAMs)
Once you have your project set up, navigate to the IAMs dashboard. You can now add project users. I strongly suggest having at least two project owners since it is in fact possible to delete your own access. I will be writing a future post on managing access to your project. I recommend adding users with the minimum access required, and ensuring you have at least one IAM manager.
Connecting important APIs
There are 100s of APIs available on GCP, some of which are automatically enabled when you set up your GCP project. Navigate to https://console.cloud.google.com/apis/, you will then be able to view all APIs currently enabled.
Click on the Enable APIs And Services button, which will open up the API library, which I strongly recommend browsing as there are some great tools which may benefit your work. However, be sure to check the pricing tab on the API info.
I suggest enabling the google drive API if you are using Google Workspace, and the BigQuery data transfer API, if you want to use tools such as scheduled codes.
You are now ready to start your journey on the GCP, once you start to get used to GCP you will find it highly accessible and a fantastic tool for working with your data. I’ll be posting blogs in the coming weeks on adding Google Analytics 4 data into BigQuery alongside advice on managing data warehouses on GCP.
