Admin Console Logs in BigQuery

Education Standard and Education Plus editions can export Google Workspace logs and usage reports to BigQuery. This feature allows schools to analyze and get insight into activity across their domain without being limited to the default expiration times for Admin console data.

About BigQuery with Google Workspace for Education

Getting Started with BigQuery 

Use the articles below to learn more about using BigQuery. 

Troubleshooting and FAQs

Do I need to purchase Google Cloud Platform (GCP) if I want to leverage BigQuery?

You can get started for free using BigQuery Sandbox. Once you have evaluated the benefits for your organization you will need to set up billing. If you are an CDW Amplified for Education customer, please contact your account manager for more information.

Is exported data retained longer than Admin console data? How long?

The retention period depends on how you configure your data exports with BigQuery. If you set up a BigQuery export using Sandbox, you may notice that only the last 60 days of data are kept.

How can I verify my data retention time in BigQuery?

  1. Login to https://console.cloud.google.com/bigquery with the account that manages your BigQuery project.
  2. Go to the project where your data is stored.
  3. Expand the project to view the datasets under it. Then expand the dataset where your Google logs live. 
    Screen_Shot_2021-08-13_at_4.33.06_PM.png
  4. Click the ellipsis to the left of the activity table and select open.
  5. Select the details tab and look for partition expiration. If there is no partition expiration listed, your data is retained indefinitely.
    Screen_Shot_2021-08-13_at_4.34.01_PM.png

How do I change the retention time for my exported data?

You'll need to change the expiration date for each table in the export: activity and usage. To change the expiration date on the activity table to one year, run the query below with your unique project id and dataset name. The same command can be run against the usage table. 

Screen_Shot_2021-08-13_at_4.36.19_PM.png

Visit this Google Cloud document for additional information.

I have Workspace Plus and use BigQuery for exporting reports, how much would the Cloud Platform project cost?

Predicting costs for exported logs and the use of those logs can be tricky. You will be billed for storage and Analysis (queries) The storage will be the cheapest and most predictable. You can use Sandbox for 60 days to see how much data gets dumped each month. So if there is 1GB of data after 60 days, you know that you can store up to a year of logs and stay under the 10GB Free storage in BigQuery. The query costs can vary a lot depending on how you are accessing the data. Data Studio will refresh every 12 hours, so that means a query of your BigQuery data every 12 hours. There is a new Data Studio feature now called Extract Data. You can take any data source, including BigQuery and extract the data once per week or once per month. This would help keep the query cost down. 

Additional Help with BigQuery

CDW Amplified for Education customers with a support contract can reach out to their account manager to schedule a BigQuery Support Stack.

 

 

 

Articles in this section