In today’s educational landscape, data-driven decision-making is key to fostering a thriving learning environment. School administrators and teachers use data to shape strategies that enhance student outcomes. At CDW Amplified for Education, we empower schools and partners to develop actionable roadmaps to achieve their goals using Google for Education tools. This article will guide you through exporting Google Workspace logs to BigQuery, ensuring your institution can retain and analyze valuable data beyond the limited time frame offered by the Google Admin console.
What You Will Learn
By the end of this article, you will understand how to set up and export Google logs to BigQuery, enabling long-term data retention, facilitating advanced analysis, and supporting data-driven decisions in your school or district.
Unlocking the Power of BigQuery
Google Workspace's Admin console typically retains data logs for only 30 to 180 days, depending on the service. This short retention period can limit the ability of school leaders to analyze and utilize their data effectively. BigQuery, a powerful data warehousing solution on the Google Cloud Platform (GCP), allows Google Workspace organizations to export and store logs for a more extended period, well beyond the limitations of the Admin console. With BigQuery, you can export and store various logs, including email logs, for advanced querying and analysis.
This capability is especially beneficial for schools using data visualization platforms like Looker Studio. The data exported to BigQuery can be seamlessly connected to these platforms, providing not only real-time but also historical insights. This comprehensive view of the data can equip school leaders with the information they need to make informed decisions, making them feel more prepared and in control.
Setting Up for Success
Before exporting data to BigQuery, you must set up billing for the Google Cloud Platform and create a new BigQuery project. After satisfying this prerequisite, the process of exporting your data is straightforward:
- Navigate to Reports: In the Google Admin console, go to Reports > Audits.
- Select BigQuery Export: Choose the option to export your logs to BigQuery.
After log export, you’ll find it organized into two primary tables:
- Activity Table: This table logs user actions, such as logins, Drive sharing, or participation in Google Meet.
- Usage Table: This table contains aggregate data, including Drive storage usage per user, Classroom activity metrics, and the number of Meets organized.
These tables, filled with data exported to BigQuery, are invaluable for monitoring trends, assessing resource usage, and informing strategic planning. They ensure that the data you rely on does not expire prematurely, providing a wealth of insights for your organization.
Managing Costs and Storage
BigQuery offers a flexible pricing model that can accommodate the needs of educational institutions. Schools receive 10GB of free storage for their exported logs monthly and 1TB of free queries. This allowance is sufficient for many schools, with carefully managed data retention policies to control costs. Even medium-sized school districts can perform weekly data queries without incurring additional charges, making BigQuery a cost-effective solution for extended data retention and analysis.
Important Considerations
When implementing BigQuery, keep the following points in mind:
- Cost Structure: Initial costs include storage and computation up to a predefined limit.
- Data Collection: BigQuery only begins collecting data from the day it is activated; it does not retroactively collect past data.
- Testing: Districts can set up a test project without immediate billing. The first Google Cloud project includes $300 in credit with a valid credit card on file.
BigQuery is poised to transform how schools manage and utilize their data. By extending the lifespan of logs and enabling administrators and educators to harness data for informed decision-making, it's a game-changer for student success.
Next Steps
To learn more about exporting Google logs to BigQuery, contact your Google Customer Success Specialist. They can assist you with setting up a BigQuery project, configuring the export of admin and Gmail logs, and using Looker Studio to explore and visualize your data.
If you currently have hours on a support subscription, you are one step closer to Support Stacks and you can connect with the support team by Emailing support@amplifiedit.cdw.com. To purchase a support contract, please complete this form, and an account manager will contact you.
| Document Version | Date | Description of Change |
| 1.0 | 3/18/2024 | Updated reference of RAM to GCSS |
| 1.1 | 5/28/2024 | removed author block |
| 1.2 | 8/27/2024 | Rewrite, reverify |