Education Standard and Education Plus editions can export Google Workspace logs and usage reports to BigQuery. This feature allows schools to analyze and get insight into activity across their domain without being limited to the default expiration times for Admin console data.
About BigQuery with Google Workspace for Education
Getting started with BigQuery
Use the articles below to learn more about using BigQuery.
- Google Workspace logs and BigQuery
- About Reporting logs and BigQuery
- Recording of CDW Amplified for Education's BigQuery Training Webinar
- Getting Started with BigQuery Logs
Troubleshooting and FAQs
Do I need to purchase Google Cloud Platform (GCP) if I want to leverage BigQuery?
Previously you could get started for free using BigQuery Sandbox. In August 2022 Google removed the separate Gmail log export to BigQuery and began including Gmail logs in the same project with other Workspace Admin log exports. It's unclear at this time if using the Sandbox to get started will continue to be an option. If you are an CDW Amplified for Education customer and would like to purchase GCP usage for log exports to BigQuery, please contact your account manager for more information.
Is exported data retained longer than Admin console data? How long?
The retention period depends on how you configure your data exports with BigQuery. If you set up a BigQuery export using Sandbox, you may notice that only the last 60 days of data are kept.
How can I verify my data retention time in BigQuery?
- Login to https://console.cloud.google.com/bigquery with the account that manages your BigQuery project.
- Go to the project where your data is stored.
- Expand the project to view the datasets under it. Then expand the dataset where your Google logs live.
- Click the ellipsis to the left of the activity table and select open.
- Select the details tab and look for partition expiration. If there is no partition expiration listed, your data is retained indefinitely.
How do I change the retention time for my exported data?
You'll need to change the expiration date for each table in the export: activity and usage. To change the expiration date on the activity table to one year, run the query below with your unique project id and dataset name. The same command can be run against the usage table.
Visit this Google Cloud document for additional information.
I have Workspace Plus and use BigQuery for exporting reports, how much would the Cloud Platform project cost?
Predicting costs for exported logs and the use of those logs can be tricky. You will be billed for storage and Analysis (queries) The storage will be the cheapest and most predictable. You can use Sandbox for 60 days to see how much data gets dumped each month. So if there is 1GB of data after 60 days, you know that you can store up to a year of logs and stay under the 10GB Free storage in BigQuery. The query costs can vary a lot depending on how you are accessing the data. Data Studio will refresh every 12 hours, so that means a query of your BigQuery data every 12 hours. There is a new Data Studio feature now called Extract Data. You can take any data source, including BigQuery and extract the data once per week or once per month. This would help keep the query cost down.
Additional Help with BigQuery
CDW Amplified for Education customers with a support contract can reach out to their account manager to schedule a BigQuery Support Stack.