Google Compute Engine

Google Compute Engine (GCE) is the Infrastructure as a Service (IaaS) component of Google Cloud Platform which is built on the global infrastructure that runs Google's search engine, Gmail, YouTube and other services. Google Compute Engine enables users to launch virtual machines (VMs) on demand. VMs can be launched from the standard images or custom images created by users. GCE users must authenticate based on OAuth 2.0 before launching the VMs. Google Compute Engine can be accessed via the Developer Console, RESTful API or command-line interface (CLI).

Google Compute Engine
Google Compute Engine
Google Compute Engine
Original author(s)Google, Inc.
Developer(s)Google
Initial releaseJune 28, 2012
Operating system
Available inEnglish
TypeVirtual Private Server
LicenseProprietary software
Websitecloud.google.com/compute/

History

Google announced Compute Engine on June 28, 2012 at Google I/O 2012 in a limited preview mode. In April 2013, GCE was made available to customers with Gold Support Package. On February 25, 2013, Google announced that RightScale was their first reseller.[1] During Google I/O 2013, many features including sub-hour billing, shared-core instance types, larger persistent disks, enhanced SDN based networking capabilities and ISO/IEC 27001 certification got announced. GCE became available to everyone on May 15, 2013. Layer 3 load balancing came to GCE on August 7, 2013. Finally, on December 2, 2013, Google announced that GCE is generally available. It also expanded the OS support, enabled live migration of VMs, 16-core instances, faster persistent disks and lowered the price of standard instances.

At the Google Cloud Platform Live event on March 25, 2014, Urs Hölzle, Senior VP of technical infrastructure announced sustained usage discounts, support for Microsoft Windows Server 2008 R2, Cloud DNS and Cloud Deployment Manager. On May 28, 2014, Google announced optimizations for LXC containers along with dynamic scheduling of Docker containers across a fleet of VM instances.[2]

Google Compute Engine Unit

Google compute engine unit (GCEU), which is pronounced as GQ, is an abstraction of computing resources. According to Google, 2.75 GCEUs represent the minimum power of one logical core (a hardware hyper-thread) based on the Sandy Bridge platform.

Persistent disks

Every Google Compute Engine instance starts with a disk resource called persistent disk. Persistent disk provides the disk space for instances and contains the root filesystem from which the instance boots. Persistent disks can be used as raw block devices. By default, Google Compute Engine uses SCSI for attaching persistent disks. Persistent Disks provide straightforward, consistent and reliable storage at a consistent and reliable price, removing the need for a separate local ephemeral disk. Persistent disks need to be created before launching an instance. Once attached to an instance, they can be formatted with the native filesystem. A single persistent disk can be attached to multiple instances in read-only mode. Each persistent disk can be up to 10TB in size. Google Compute Engine encrypts the persistent disks with AES-128-CB, and this encryption is applied before the data leaves the virtual machine monitor and hits the disk. Encryption is always enabled and is transparent to Google Compute Engine users. The integrity of persistent disks is maintained via a HMAC scheme.

On June 18, 2014, Google announced support for SSD persistent disks. These disks deliver up to 30 IOPS per GB which is 20x more write IOPS and 100x more read IOPS than the standard persistent disks.

Images

An image is a persistent disk that contains the operating system and root file system that is necessary for starting an instance. An image must be selected while creating an instance or during the creation of a root persistent disk. By default, Google Compute Engine installs the root filesystem defined by the image on a root persistent disk. Google Compute Engine provides CentOS and Debian images as standard Linux images. Red Hat Enterprise Linux (RHEL) and Microsoft Windows Server 2008 R2 images are a part of the premier operating system images which are available for an additional fee. CoreOS, the lightweight Linux OS based on Chromium OS is also supported on Google Compute Engine.

Machine types

Google Compute Engine uses KVM as the hypervisor,[3] and supports guest images running Linux and Microsoft Windows which are used to launch virtual machines based on the 64 bit x86 architecture. VMs boot from a persistent disk that has a root filesystem. The number of virtual CPUs, amount of memory supported by the VM is dependent on the machine type selected.

Billing and discounts

Once an instance is run for over 25% of a billing cycle, the price starts to drop:

  • If an instance is used for 50% of the month, one will get a 10% discount over the on-demand prices
  • If an instance is used for 75% of the month, one will get a 20% discount over the on-demand prices
  • If an instance is used for 100% of the month, one will get a 30% discount over the on-demand prices

Machine type comparison

Google provides certain types of machine:

  • Standard machine: 3.75 GB of RAM per virtual CPU
  • High-memory machine: 6.5 GB of RAM per virtual CPU
  • High-CPU machine: 0.9 GB of RAM per virtual CPU
  • Shared machine: CPU and RAM are shared between customers
  • Memory-optimized machine: greater than 14 GB RAM per vCPU.

The prices mentioned below[4] are based on running standard Debian or CentOS Linux virtual machines (VMs). VMs running proprietary operating systems will be charged more.

Machine type Machine name Virtual cores Memory Cost per hour (US hosted) Cost per hour (Europe hosted)
Standard n1-standard-1 1 3.75GB $0.070 $0.077
Standard n1-standard-2 2 7.5GB $0.140 $0.154
Standard n1-standard-4 4 15GB $0.280 $0.308
Standard n1-standard-8 8 30GB $0.560 $0.616
Standard n1-standard-16 16 60GB $1.120 $1.232
High Memory n1-highmem-2 2 13GB $0.164 $0.180
High Memory n1-highmem-4 4 26GB $0.328 $0.360
High Memory n1-highmem-8 8 52GB $0.656 $0.720
High Memory n1-highmem-16 16 104GB $1.312 $1.440
High CPU n1-highcpu-2 2 1.80GB $0.088 $0.096
High CPU n1-highcpu-4 4 3.60GB $0.176 $0.192
High CPU n1-highcpu-8 8 7.20GB $0.352 $0.384
High CPU n1-highcpu-16 16 14.40GB $0.704 $0.768
Shared Core f1-micro 0.2 0.60GB $0.013 $0.014
Shared Core g1-small 0.5 1.70GB $0.035 $0.0385
Memory-optimized n1-ultramem-40 40 938GB $6.3039 $6.9389
Memory-optimized n1-ultramem-80 80 1922GB $12.6078 $13.8779
Memory-optimized n1-megamem-96 96 1433.6GB $10.6740 $11.7430
Memory-optimized n1-ultramem-160 160 3844GB $25.2156 $27.7557

Resources

Compute Engine connects various entities called resources that will be a part of the deployment. Each resource performs a different function. When a virtual machine instance is launched, an instance resource is created that uses other resources, such as disk resources, network resources and image resources. For example, a disk resource functions as data storage for the virtual machine, similar to a physical hard drive, and a network resource helps regulate traffic to and from the instances.

Image

An image resource contains an operating system and root file system necessary for starting the instance. Google maintains and provides images that are ready-to-use or users can customize an image and use that as an image of choice for creating instances. Depending on the needs, users can also apply an image to a persistent disk and use the persistent disk as the root file system.

Machine type

An instance's machine type determines the number of cores, the memory, and the I/O operations supported by the instance.

Disk

Persistent disks are independent of the virtual machines and outlive an instance's lifespan. All information stored on the persistent disks is encrypted before being written to physical media, and the keys are tightly controlled by Google.

Type Price (per GB/month)
Standard provisioned space $0.04
SSD provisioned space $0.17
Snapshot storage $0.026
IO operations No additional charge

Each instance can attach only a limited amount of total persistent disk space (one can have up to 64 TB on most instances) and a limited number of individual persistent disks (one can attach up to 16 independent persistent disks to most instances).

Snapshot

Persistent disk snapshots lets the users copy data from existing persistent disk and apply them to new persistent disks. This is especially useful for creating backups of the persistent disk data in cases of unexpected failures and zone maintenance events.

Instance

A Google Compute Engine instance is a virtual machine running on a Linux or Microsoft Windows configuration. Users can choose to modify the instances including customizing the hardware, OS, disk, and other configuration options.

Network

A network defines the address range and gateway address of all instances connected to it. It defines how instances communicate with each other, with other networks, and with the outside world. Each instance belongs to a single network and any communication between instances in different networks must be through a public IP address.

Your Cloud Platform Console project can contain multiple networks, and each network can have multiple instances attached to it. A network allows you to define a gateway IP and the network range for the instances attached to that network. By default, every project is provided with a default network with preset configurations and firewall rules. You can choose to customize the default network by adding or removing rules, or you can create new networks in that project. Generally, most users only need one network, although you can have up to five networks per project by default.

A network belongs to only one project, and each instance can only belong to one network. All Compute Engine networks use the IPv4 protocol. Compute Engine currently does not support IPv6. However, Google is a major advocate of IPv6 and it is an important future direction.

Address

When an instance is created, an ephemeral external IP address is automatically assigned to the instance by default. This address is attached to the instance for the life of the instance and is released once the instance has been terminated. GCE also provides mechanism to reserve and attach static IPs to the VMs. An ephemeral IP address can be promoted to a static IP address.

Firewall

A firewall resource contains one or more rules that permit connections into instances. Every firewall resource is associated with one and only one network. It is not possible to associate one firewall with multiple networks. No communication is allowed into an instance unless a firewall resource permits the network traffic, even between instances on the same network.

Route

Google Compute Engine offers a routing table to manage how traffic destined for a certain IP range should be routed. Similar to a physical router in the local area network, all outbound traffic is compared to the routes table and forwarded appropriately if the outbound packet matches any rules in the routes table.

Regions and zones

A region refers to a geographic location of Google's infrastructure facility. Users can choose to deploy their resources in one of the available regions based on their requirement. As of June 1, 2014, Google Compute Engine is available in central US region, Western Europe and Asia East region.

A zone is an isolated location within a region. Zones have high-bandwidth, low-latency network connections to other zones in the same region. In order to deploy fault-tolerant applications that have high availability, Google recommends deploying applications across multiple zones in a region. This helps protect against unexpected failures of components, up to and including a single zone. As of August 5, 2014, there are eight zones - three each in central US region and Asia East region and two zones in Western Europe region.

Scope of resources

All resources within GCE belong to the global, regional, or zonal plane. Global resources are accessible from all the regions and zones. For example, images are a global resource so users can launch a VM in any region based on a global image. But an address is a regional resource that is available only to the instances launched in one of the zones within the same region. Instances are launched in a specific zone that requires the zone specification as a part of all requests made to that instance.

The table below summarises the scope of GCE resources:

Scope Resource
Global Image
Global Snapshot
Global Network
Global Firewall
Global Route
Region Address
Zone Instance
Zone Machine Type
Zone Disk

Features

Billing and pricing model

Google charges the VMs for a minimum of 10 minutes. At the end of 10th minute, instances are charged in 1-minute increments, rounded up to the nearest minute.[5] Sustained usage based pricing will credit the discounts to the customers based on the monthly utilisation.[6][7] Users need not pay a commitment fee upfront to get discounts on the regular, on-demand pricing.

VM performance

Compute Engine VMs boot within 30 seconds[8] which is considered to be 4-10x faster than the competition.

Disk performance

The persistent disks of Compute Engine deliver higher IOPS consistently.[9] With the cost of provisioned IOPS included within the cost of storage, users need not pay separately for the IOPS.[10]

Global scope for images and snapshots

Images and disk snapshots belong to the global scope which means they are implicitly available across all the regions and zones of Google Cloud Platform.[11] This avoids the need for exporting and importing images and snapshots between regions.

Transparent maintenance

During the scheduled maintenance of Google data center, Compute Engine can automatically migrate the VMs from one host to the other without involving any action from the users. This delivers better uptime to applications.[12][13]

References

  1. ^ Barb Darrow (February 25, 2013). "Exclusive: RightScale is first to resell, support Google Compute Engine". Gigaom blog. Retrieved July 5, 2013.
  2. ^ Google (June 10, 2014). "Containers on Google Cloud Platform". Google Compute Engine documentation. Retrieved June 10, 2014.
  3. ^ Metz, Cade (3 July 2012). "Google Shaman Explains Mysteries of 'Compute Engine'". Wired. Retrieved 26 September 2016.
  4. ^ "Google Compute Engine Pricing". Google Cloud Platform. Retrieved 2018-02-19.
  5. ^ Buttler, Brandon. "Google rolls out by-the-minute cloud billing, introduces a new NoSQL database". NetworkWorld. Archived from the original on 30 July 2013. Retrieved 6 April 2014.
  6. ^ Joneja, Navneet. "Introducing Sustained Use Discounts - Automatically pay less for sustained workloads on Compute Engine". Google Cloud Blog. Google. Retrieved 6 April 2014.
  7. ^ Babcock, Charles (2014-03-28). "Google Wins In Amazon Cloud Price Battle". InformationWeek. Retrieved 6 April 2014.
  8. ^ Staddill, Sebastian. "By the numbers: How Google Compute Engine stacks up to Amazon EC2". Gigaom. Gigaom. Retrieved 6 April 2014.
  9. ^ Stadill, Sebastian. "Scalr Cloud Benchmarks". Scalr Cloud Benchmarks. Scalr. Archived from the original on 29 March 2014. Retrieved 6 April 2014.
  10. ^ "Persistent Disk Pricing". Google Compute Engine Documentation. Google. Retrieved 6 April 2014.
  11. ^ "Global Resources". Google Compute Engine Documentation. Google. Retrieved 6 April 2014.
  12. ^ Adler, Brian. "Google Compute Engine Live Migration Passes the Test". RightScale Blog. RightScale. Retrieved 6 April 2014.
  13. ^ Leong, Lydia. "Google Compute Engine and live migration". Blog.

External links

Abiquo Enterprise Edition

Abiquo Hybrid Cloud Platform is a web-based cloud computing software platform developed by

Abiquo. Written entirely in Java, it is used to build, integrate and manage public and private clouds in homogeneous environments. Users can deploy and manage servers, storage system and network and virtual devices. It also supports LDAP integration.

Alluxio

Alluxio is an open-source virtual distributed file system (VDFS). Initially as research project "Tachyon", Alluxio was created at the University of California, Berkeley's AMPLab as Haoyuan Li's Ph.D. Thesis, advised by Professor Scott Shenker & Professor Ion Stoica. Alluxio sits between computation and storage in the big-data analytics stack. It provides a data abstraction layer for computation frameworks, enabling applications to connect to numerous storage systems through a common interface. Alluxio's memory-centric architecture enables data access at speeds orders of magnitude faster than existing harddisk-based solutions. The software is published under the Apache License.

Alluxio can be deployed on-premise, in the cloud (e.g. Microsoft Azure, AWS, Google Compute Engine), or a hybrid cloud environment. It can run on bare-metal or in a containerized environments such as Kubernetes, Docker, Apache Mesos.

Apache Ignite

Apache Ignite is an open-source distributed database, caching and processing platform designed to store and compute on large volumes of data across a cluster of nodes.Ignite was open-sourced by GridGain Systems in late 2014 and accepted in the Apache Incubator program that same year. The Ignite project graduated on September 18, 2015.Apache Ignite's database utilizes RAM as the default storage and processing tier, thus, belonging to the class of in-memory computing platforms. The disk tier is optional but, once enabled, will hold the full data set whereas the memory tier will cache full or partial data set depending on its capacity.

Regardless of the API used, data in Ignite is stored in the form of key-value pairs. The database component scales horizontally, distributing key-value pairs across the cluster in such a way that every node owns a portion of the overall data set. Data is rebalanced automatically whenever a node is added to or removed from the cluster.

On top of its distributed foundation, Apache Ignite supports a variety of APIs including JCache-compliant key-value APIs, ANSI-99 SQL with joins, ACID transactions, as well as MapReduce like computations.

Apache Ignite cluster can be deployed on-premise on a commodity hardware, in the cloud (e.g. Microsoft Azure, AWS, Google Compute Engine) or in a containerized and provisioning environments such as Kubernetes, Docker, Apache Mesos, VMWare.

AppScale

AppScale is an open-source serverless computing platform that automatically deploys and scales unmodified Google App Engine applications over public and private clouds, as well as on-premises clusters. AppScale is modeled on the App Engine APIs and has support for Go, Java, PHP, and Python applications.The goal of AppScale is to provide developers with a rapid, API-driven development platform that can run applications on any cloud infrastructure. AppScale decouples app logic from its service ecosystem to give developers and cloud administrators control over app deployment, data storage, resource use, backup and migration.The platform allows developers to focus on the business logic of building scalable applications, freeing them from the need to concentrate on deployment and scaling logic. AppScale decouples app logic from its service ecosystem to give developers and cloud administrators control over app deployment, data storage, resource use, backup, and migration.AppScale includes high-level APIs for persistence, asynchronous execution, distributed memory cache, user authentication, and more. It handles service discovery, load-balancing, fault-tolerance, and auto-scaling. It also allows IT operations and cloud administrators to provide a consistent, tunable environment for running and maintaining apps on multiple cloud infrastructures.

AppScale is developed and maintained by AppScale Systems, Inc., based in Santa Barbara, California, and Google.

Autoscaling

Autoscaling, also spelled auto scaling or auto-scaling, and sometimes also called automatic scaling, is a method used in cloud computing, whereby the amount of computational resources in a server farm, typically measured in terms of the number of active servers, scales automatically based on the load on the farm. It is closely related to, and builds upon, the idea of load balancing.

BOSH (software)

BOSH is an open source project that offers a tool chain for release engineering, deployment & life-cycle management of large scale distributed services. Namely, this tool chain is made of a server (the BOSH Director) and a command line tool. BOSH is typically used to package, deploy and manage cloud software. While BOSH was initially developed by VMware in 2010 to deploy Cloud Foundry PaaS, it can be used to deploy other software (such as Hadoop, RabbitMQ, or MySQL for instance). BOSH is particularly well-suited for managing the whole life cycle of large distributed systems.

Since March 2016, BOSH can manage deployments on both Windows and Linux servers.

A BOSH Director communicates with a single IaaS that provides the underlying networking and VMs (or containers). Several IaaS providers are supported: Amazon Web Services EC2, Apache CloudStack, Google Compute Engine, Microsoft Azure, OpenStack, and VMware vSphere.

To help support more underlying infrastructures, BOSH uses a concept of Cloud Provider Interface (CPI). There is an implementation of the CPI for each of the IaaS listed above. Typically the CPI is used to deploy VMs, but it can be used to deploy containers.

Few CPIs exist for deploying containers with BOSH, and only one actively supported. For this, BOSH uses a CPI that deploys Pivotal's Garden containers (Garden is very similar to Docker) on a single virtual machine, run by VirtualBox or VMware Workstation. In theory, any other container engine could be supported, if the necessary CPIs were developed.

Due to BOSH indifferently supporting deployments on VMs or containers, BOSH uses the generic term “instances” to designate those. It is up to the CPI to choose whether a BOSH “instance” is actually a VM or a container.

Cloud database

A cloud database is a database that typically runs on a cloud computing platform, and access to the database is provided as-a-service.

Database services take care of scalability and high availability of the database. Database services make the underlying software-stack transparent to the user.

Cloudscaling

The Cloudscaling Group, Inc., was a software company based in San Francisco, California, USA. The company’s Open Cloud System is a cloud computing system based on the OpenStack open-source software project. It is used to deploy infrastructure as a service (IaaS) public, private and hybrid clouds that support applications typically found on public cloud infrastructures such as Amazon Web Services or Google Compute Engine. These applications are often referred to as cloud-ready applications, which Cloudscaling refers to as dynamic applications and are contrasted with traditional enterprise IT applications.

Cycle Computing

Cycle Computing is a company that provides software for orchestrating computing and storage resources in cloud environments. The flagship product is CycleCloud, which supports Amazon Web Services, Google Compute Engine, Microsoft Azure, and internal infrastructure. The CycleCloud orchestration suite manages the provisioning of cloud infrastructure, orchestration of workflow execution and job queue management, automated and efficient data placement, full process monitoring and logging, within a secure process flow.

GCE

GCE can mean:

General Certificate of Education

Global citizenship education

Ground combat element in the United States Marine Corps

Google Compute Engine

Guthrie Corridor Expressway, an expressway in Malaysia

Google Cloud Dataproc

Google Cloud Dataproc (Cloud Dataproc) is a cloud-based managed Spark and Hadoop service offered on Google Cloud Platform. Cloud Dataproc utilizes many Google Cloud Platform technologies such as Google Compute Engine and Google Cloud Storage to offer fully managed clusters running popular data processing frameworks such as Apache Hadoop and Apache Spark.

Google Cloud Platform

Google Cloud Platform (GCP), offered by Google, is a suite of cloud computing services that runs on the same infrastructure that Google uses internally for its end-user products, such as Google Search and YouTube. Alongside a set of management tools, it provides a series of modular cloud services including computing, data storage, data analytics and machine learning. Registration requires a credit card or bank account details.Google Cloud Platform provides Infrastructure as a service, Platform as a service, and Serverless computing environments.

In April 2008, Google announced App Engine, a platform for developing and hosting web applications in Google-managed data centers, which was the first cloud computing service from the company. The service became generally available in November 2011. Since the announcement of App Engine, Google added multiple cloud services to the platform.

Google Cloud Platform is a part of Google Cloud, which includes the Google Cloud Platform public cloud infrastructure, as well as G Suite, enterprise versions of Android and Chrome OS, and application programming interfaces (APIs) for machine learning and enterprise mapping services.

H2O (software)

H2O is open-source software for big-data analysis. It is produced by the company H2O.ai. H2O allows users to fit thousands of potential models as part of discovering patterns in data.

The H2O software runs can be called from the statistical package R, Python, and other environments. It is used for exploring and analyzing datasets held in cloud computing systems and in the Apache Hadoop Distributed File System as well as in the conventional operating-systems Linux, macOS, and Microsoft Windows. The H2O software is written in Java, Python, and R. Its graphical-user interface is compatible with four browsers: Chrome, Safari, Firefox, and Internet Explorer.

Juju (software)

Juju is an open source application modeling tool developed by Canonical Ltd. Juju focuses on reducing the operation overhead of today's software by facilitating quickly deploying, configuring, scaling, integrating, and performing operational tasks on a wide choice of public and private cloud services along with bare metal servers and local container based deployments.

LoadRunner

LoadRunner is a software testing tool from Micro Focus. It is used to test applications, measuring system behaviour and performance under load. LoadRunner can simulate thousands of users concurrently using application software, recording and later analyzing the performance of key components of the application.

LoadRunner simulates user activity by generating messages between application components or by simulating interactions with the user interface such as keypresses or mouse movements. The messages and interactions to be generated are stored in scripts. LoadRunner can generate the scripts by recording them, such as logging HTTP requests between a client web browser and an application's web server.Hewlett Packard Enterprise acquired LoadRunner as part of its acquisition of Mercury Interactive in November 2006. In Sept 2016, Hewlett Packard Enterprise announced it is selling its software business, including Mercury products, to Micro Focus. As of 01-Sept-2017, the acquisition was complete.

RightScale

RightScale is a company that sells software as a service for cloud computing management for multiple providers. The company is based in Santa Barbara, California.

SUSE

SUSE ( SOO-zə) is a German-based, multinational, open-source software company that develops and sells Linux products to business customers. Founded in 1992, it was the first company to market Linux for the enterprise. It is the primary sponsor of the community-supported openSUSE Project, which develops the openSUSE Linux distribution.

In July 2018, Micro Focus International, SUSE's parent company since 2014, announced its plan to sell the business unit to a subsidiary of EQT Partners in the first quarter of calendar year 2019. This acquisition was completed on March 15, 2019, making SUSE a standalone business.

TensorFlow

TensorFlow is a free and open-source software library for dataflow and differentiable programming across a range of tasks. It is a symbolic math library, and is also used for machine learning applications such as neural networks. It is used for both research and production at Google.‍   It is a standard expectation in the industry to have experience in TensorFlow to work in machine learning.TensorFlow was developed by the Google Brain team for internal Google use. It was released under the Apache 2.0 open-source license on November 9, 2015.

Tensor processing unit

A tensor processing unit (TPU) is an AI accelerator application-specific integrated circuit (ASIC) developed by Google specifically for neural network machine learning.

Overview
Advertising
Communication
Software
Platforms
Hardware
Development
tools
Publishing
Search
(timeline)
Events
People
Other
Related
As a service
Technologies
Applications
Platforms
Infrastructure

This page is based on a Wikipedia article written by authors (here).
Text is available under the CC BY-SA 3.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.