Mike: Applications can have various types of data. For example, if your application is a social image sharing site, you have to store image files, high volumes of user messages and transactional data. You'll likely need to cache frequently accessed data as well, and of course, you'll want to collect, query and analyze all the data to gather business intelligence about your users as well as product usage patterns. Google Cloud offers a number of managed services that you can use for each type of data. In this module, Overview of Data Storage Options, you'll learn about services such as Cloud Storage, Datastore, Cloud Bigtable, Cloud SQL, Cloud Spanner and BigQuery. You'll learn the ideal use cases for each data storage option as well as use cases for which the option might not be suitable. Armed with this knowledge, you can choose a data storage option that meets the specific use case in your application. A common problem that application developers face is handling database connections in a secure manner. You'll learn how to connect your Cloud SQL second-generation instance without having to allow-list IP addresses or configure SSL. You have a full suite of cost-effective storage services to choose from when developing with Google Cloud, no one-size-fits-all. Your choice of storage and database solutions will depend on your application and workload. Cloud Storage is a unified object storage for developers and enterprises that allows you to serve, analyze and archive data. Objects are accessed via HTTP requests including ranged gets to retrieve portions of the data. The only key is the object name. There's object metadata, but the object itself is treated as just bytes with no structure. The scale of the system allows for serving static content or accepting user-uploaded content like photos and videos. Cloud Storage is built for availability, durability, scalability and consistency. It is an ideal solution for storing images and videos, objects and blobs and any unstructured data. Firestore is a fast, fully managed, serverless, NoSQL document database built for automatic scaling, high performance and ease of application development. Firestore in native mode provides features such as a strongly consistent storage layer, a collection and document-based data model, real time updates and offline features and mobile and Web client libraries. Firestore is built to scale and takes advantage of Google Cloud's powerful infrastructure with automatic horizon scaling in and out in respond to your application's load. Security access controls for data are built in and enable you to handle data validation via a configuration language. For more information, see the Firestore documentation available in the download pane below. Firestore in Datastore mode, which is often referred to as Datastore, is a highly scalable NoSQL database for your applications. Datastore automatically handles sharding and replication providing you with a highly available and durable database that scales automatically to handle your application's load. Datastore provides a myriad of capabilities such as asset transactions, SQL-like queries, indexes and much more. Datastore scales seamlessly and automatically with your data, allowing applications to maintain high performance as they receive more traffic. Datastore is an ideal solution when your application data is semistructured or hierarchical and when you need to store durable key value data at scale. Typical use cases for Datastore include product catalogs, user profiles and transactions based on asset properties. Datastore is not ideal for every use case. For example, Datastore is not a relational database, and it is not an effective solution for analytic data. Datastore was formerly implemented as an extension of Cloud Bigtable but now is implementing using Firestore. Firestore in Datastore mode is backward compatible with Datastore, but the native mode's data model, real time updates and mobile and Web client library features are not available when using Datastore mode. Firestore in Datastore mode provides improvements to the original Datastore including strongly consistent queries, transaction improvements and the removal of the one-per-second limit on rights to an entity group. Eventually, existing Datastore instances will be automatically migrated to Firestore in Datastore mode. No functional changes are required to existing Datastore code in applications. Bigtable is a high-performance NoSQL database service. It's a sparsely populated table that can scale to billions of rows and thousands of columns. Bigtable can store terabytes to petabytes of data. Bigtable is built for fast key-value lookup and scanning over a defined key range. It's similar to a spreadsheet that gives you access to any set of columns from contiguous rows by searching only the value in the first column, or the key. Updates to individual rows are atomic. Due to the fast lookup and write speed of Bigtable, it's great for user behavior. Bigtable supports operational and analytical applications and is ideal for storing large amounts of single-key data and MapReduce operations. Bigtable offers consistent sub-10-millisecond latency. Bigtable offers seamless scaling. Changes to the deployment configuration are immediate, so there's no downtime during reconfiguration. Cloud Bigtable supports the open-source industry standard HBase API. Cloud SQL is Google Cloud's relational database service. It is a managed service that lets Google manage replication, failover and backups of your databases so you can focus on your MySQL, PostgreSQL or SQL-Server-compatible applications. Cloud SQL lets you easily configure replication and backups to protect your data. You can replicate a primary instance to one or more read replicas. A read replica is a copy of the primary that reflects changes to the primary instance in almost real time. You can enable automatic failover to make your database highly available. Backups allow you to restore your Cloud SQL instance, to recover lost data or recover from a problem with your instance. You can enable automated backups for any instance that contains data that you need to protect from loss or damage. The Cloud SQL proxy works by having a local client called the proxy running in the local environment. Your application communications with the proxy with the standard database protocol used by your database. The proxy uses a secure tunnel to communicate with its companion process running on the server. Cloud SQL proxy provides secure access to your Cloud SQL second-generation instances without your having to allow IP addresses or configure SSL. The proxy uses the Cloud SQL API to authenticate with Google Cloud. You must enable the API before using the proxy, and you must provide the proxy with a valid user account. Cloud SQL is ideal for Web frameworks, applications requiring structured data and online transaction processing, OLTP, workloads. It is ideal for applications using MySQL, PostgreSQL or SQL Server with minimal refactoring required for migration to Google Cloud. Cloud Spanner is Google Cloud's fully managed relational database service offering both strong consistency and horizontal scalability. It is designed for mission-critical OLTP applications. Cloud Spanner provides automatic synchronous replication for high availability. Spanner is built for multiregion replication and offers one of the highest SLAs in the industry, 99.999 percent. Spanner is ideal for applications with relational, structured and semistructured data that require high availability, strong consistency and transactional reads and writes. Unlike Cloud SQL, Spanner requires every table to have a primary key. Another difference is that Spanner also supports interleaf tables where child rows are inserted into the table adjacent to the parent row. This improves the query performance when joins are done Bigtable parent and child.
