+1-617-500-8286USA
+91-832-950-1237INDIA

Sea Change Series: Scale in the Mega Data Center

  • ID 125401
  • May 2017
  • 102 pages
  • WinterGreen Research

Next generation mega data center technology is able to leverage scale to implement cloud computing that is better than most of what is out there now.

Scale is a vital part of the technology used to support next generation data centers. The study is targeted to C-level executives that need to move quickly and surely to improve IT. Automation of IT depends on understanding the business market opportunity from an independent perspective. Vendors are smart but they are committed to the technology they are pushing, the Sea Change Series is able to provide a perspective not available anywhere else.

Extreme scale is what brings enough pathways inside a Mega data center to create a non-blocking (CLOS) networked server architecture. Non-blocking network architecture benefits the business because it permits launching thousands of virtual severs on demand at the application layer. In this manner, innovation can be made to happen quickly.

Using a mega data center, DevOps and/or automated processes can request and deploy additional resource without backing the Dell truck up to the data center every week to provide on-demand capacity. Automated deprovisioning handles freeing of surplus resources, while being virtual means not having stacks of surplus hardware to dispose of as underutilized capital assets.

Modern data centers are organized into processing nodes that manage different applications at a layer above infrastructure. Data is stored permanently and operated on in place. These are the two technologies to check for when choosing a data center. These architectural features provide economies of scale that greatly reduce the IT spend while offering better quality IT.

Once scale is in place, then the economies of scale kick in. When negotiating for cloud capability, managers need to check to see that sufficient multiple pathways are available to reach any node in a non-blocking manner. Non-blocking architecture is more efficient than other IT infrastructure and supports better innovation for apps and smart digitization. Not all cloud architectures offer this business benefit.

The Cloud 2.0 mega data center platform fabric technologies support the digital economy by creating scale with a network internal to the mega data center system image. Without scale, there are not enough pathways inside the data center to enable elimination of bottlenecks. Access to every node in the fabric, and multiple duplicate pathways to every node are needed to enable real time application connectivity. With sufficient scale, if one pathway is blocked, there are enough other pathways to get to the desired node resources.

The theme of this study is that scale matters.

Scale can be implemented by IT, but the executive needs to understand that there is a difference between different technologies. In a mega data center, the system is implemented as a fabric: servers are linked to top of rack switches, which are made from merchant silicon chips, mostly Broadcom, some switch ASICs. A pod of server racks are linked to each other through an edge aggregation switch. A pod of server racks is based on the same ASICs as in the top of rack switches.

The importance of nonblocking architecture is compelling. Aggregation switches are lashed together through a set of non-blocking spine switches. All switches are based on the same chip. This is precisely the way Facebook is building its own Wedge and 6-pack open switches- nine years after Google did it. The amazing thing is that Facebook had not done this already.

The four superstar companies that are able to leverage IT to achieve growth, Microsoft, Google, Facebook, and the leader AWS all use Clos architecture. What is significant is that systems have to hit a certain scale before Clos networks work Clos networks are what work now for flexibility and supporting innovation in an affordable manner. There is no dipping your toe in to try the system to see if it will work, it will not and then the IT says, “We tried that, we failed,” but what the executive needs to understand is that scale matters. A little mega data center does not exist. Only scale works.

Maybe scale is not the only answer, maybe in 20 years, Quantum computing will bring a new data center system, but for now, Clos architecture and scale dominate those IT centers that have the strongest growth engine.

Business leaders are challenged to move their enterprises to the next level of competition. An effective digital business player, transformer, and disruptor position depends on the effectiveness of employing digital technologies and leveraging connected digital systems. Organizational, operational, and business model innovation are needed to create ways of operating and growing the business using mega data center cloud technologies, systems are evolving. It is a journey to achieve the connected enterprise, ultimately connecting all employees and a trillion connected devices.

Many companies are using digital technology to create market disruption. Amazon, Uber, Google, IBM, and Microsoft represent companies using effective disruptive strategic positioning. As entire industries shift to the digital world, once buoyant companies are threatened with disappearing. A digital transformation represents an approach that enables organizations to drive changes in their business models and ecosystems leveraging cloud computing, and not just hyperscale systems but leveraging mega data centers. Just as robots make work more automated, so also cloud based communications systems implement the IoT digital connectivity transformation.

Disruption in the business markets represents major opportunity for vendors with cloud offerings. This is part of a larger digital transformation, a digital approach to interconnecting everything that enables organizations to drive changes in their business models and ecosystems.

Disruptive cloud systems are provided by Amazon (AWS), Microsoft, Google, and Facebook. Data centers are in a class by themselves, they have functioning fully automatic, self-healing, networked mega datacenters that operate at fiber optic speeds to create a fabric that can access any node in any particular data center because there are multiple pathways to every node. In this manner, they automate applications integration for any data in the mega data center.

By leveraging digital competencies, businesses can grow faster than they would otherwise. A digital strategy, in conjunction with the appropriate unified communications solution permits the implementation of innovative communications services. Digital connectivity with combined voice, video and file transfer can help organizations and their end users innovate and compete more effectively. It is imperative that organizations have a digital communications strategy in place.

This is an era where the distinction between the technologies and processes that businesses deploy is tightly linked. Digital technology directly impacts customers and markets. The boundary between internal operations of the enterprise and its external ecosystem is rapidly disappearing. Customers, markets, competitors, partners, and regulators are inextricably linked.

“Mega data centers need to be understood by all senior executives whether they move in that direction or not. These are the IT used by the fastest growing organizations Google, AWS, Microsoft, and Facebook. There are 25 Sea Change Data Center study modules describing different aspects of the move to mega data centers. The Scale module describes that it is not sufficient just to try certain cloud techniques."

“Scale is an essential aspect of the data center positioning for these leading companies. These companies use Clos networks as their data center implementations. This module addresses how and why scale in the mega data center is important. The market shift to non-blocking network inside data center building means companies have to hit a certain scale before Clos networks work.”

Key Topics:

Scale In The Mega Data Center
Realign IT Cost Structure
Mega Datacenter Physical Infrastructure
Automation of Mega Data Center
Networking Fabric
Exchange Of Data Between Servers
Complex Automation Of Process
Applications Customized For Each User
Machine-To-Machine Management of Traffic Growth
Fabric Network Topology
Building-Wide Connectivity
Highly Modular Data Cebter Design
Scale Capacity
Back-End Service Tiers
Applications Scaling
Mega Data Center Network
Fabric Next-Generation Data Center Network Design
Pod Unit of Network
Mega Data Center Server Pods
Non-Blocking Network Architecture
Data Center Auto Discovery
Large-Scale Network
Rapid Deployment Architecture
Expedites Provisioning And Changes
Programmable Access To Network Stack
Software Defined Networking (SDN)-Supports Scale and Automation
Compute Engine Load Balancing
Load Balanced Requests Architecture
Scale-Out: Server And Storage Expansion
Switches and Routers Deployed in Fabrics
Mega Data Center Multipathing
Routing Destinations
Clos Topology Network
Capacity Scalability
Aggregation Switches
Intelligent Cloud Platform
Linux For Azure

Sea Change Series: Scale in the Mega Data Center Executive Summary

Amazon, Google, Microsoft, Facebook
IT Is Better When New Sets Of Servers Can Be Spun With The Push Of A Button
Aim to Realign IT Cost Structure
Scale Matters
Table of Contents
1. Effect of Scale in the Mega Data Center
1.1 Facebook Mega Datacenter Physical Infrastructure
1.1.1 Facebook Automation of Mega Data Center Process
1.1.2 Facebook Altoona Data Center Networking Fabric
1.1.3 Facebook Altoona Cloud Mega Data Center
1.1.4 Facebook Altoona Data Center Innovative Networking Fabric Depends on Scale
1.1.5 Facebook Fabric Operates Inside the Data Center
1.1.6 Facebook Fabric
1.1.7 Exchange Of Data Between Servers Represents A Complex Automation Of Process

2. Applications Customized For Each User
2.1 Customized Back-End Service Tiers And Applications
2.2 Machine-To-Machine Management of Traffic Growth
2.2.1 Facebook Data Center Fabric Network Topology
2.2.2 Building-Wide Connectivity
2.3 Highly Modular Design Allows Users To Quickly Scale Capacity In Any Dimension
2.3.1 Back-End Service Tiers And Applications
2.3.2 Scaling Up As a Basic Function Of The Mega Data Center Network
2.3.3 Facebook Fabric Next-Generation Data Center Network Design: Pod Unit of Network

3. Mega Data Center Server Pods
3.1 Server Pods Permit An Architecture To Implement High-Performance Connectivity
3.1.1 Facebook Sample Pod: Unit of Network
3.1.2 Non-Blocking Network Architecture
3.2 Data Center Auto Discovery
3.3 Facebook Large-Scale Network
3.3.1 Rapid Deployment Architecture
3.3.2 Facebook Expedites Provisioning And Changes

4. Google Mega Data Center Scale
4.1 Google Douglas County Mega Data Center
4.1.1 Google Data Center Efficiency Measurements
4.1.2 Google Programmable Access To Network Stack
4.1.3 Google Software Defined Networking (SDN)-Supports Scale and Automation
4.1.4 Google Compute Engine Load Balancing
4.1.5 Google Compute Engine Load Balanced Requests Architecture
4.1.6 Google Compute Engine Load Balancing Scaling
4.2 Google Switches Provide Scale-Out: Server And Storage Expansion
4.2.1 Google Uses Switches Deployed in Fabrics
4.2.2 Google Mega Data Center Multi-pathing
4.2.3 Google Mega Data Center Multipathing: Routing Destinations
4.2.4 Google Clos Topology Network Capacity Scalability
4.2.5 Google Aggregation Switches Are Lashed Together Through a Set Of Non-Blocking Spine Switches
4.3 Google Network Called Jupiter

5. Microsoft Mega Data Center Scale
5.1 Microsoft Cloud Data Center Multi-Tenant Containers
5.2 Microsoft Azure Running Docker Containers
5.2.1 Microsoft Data Center, Dublin, 550,000 Sf
5.2.2 Microsoft Builds Intelligent Cloud Platform
5.3 Microsoft Server Products And Cloud Services
5.3.1 Microsoft Crafts Homegrown Linux For Azure Switches
5.3.2 Microsoft Azure Has Scale
5.4 Microsoft Azure Stack Hardware Foundation
5.5 Microsoft Azure Stack Key Systems Partners: Cisco Systems, Lenovo, Fujitsu, and NEC
5.6 Microsoft Gradual Transformation From A Platform Cloud To A Broader Offering Leveraging Economies of Scale
5.7 Microsoft Contributing to Open Systems
5.8 Microsoft Mega Data Center Supply Chain
5.9 Microsoft Leverages Open Compute Project to Bring Benefit to Enterprise Customers
5.9.1 Microsoft Assists Open Compute to Close The Loop On The Hardware Side
5.10 Microsoft Project Olympus Modular And Flexible
5.11 Microsoft Azure
5.11.1 Microsoft Azure Active Directory Has Synchronization
5.12 Microsoft Azure Has Scale

6. Mega Data Center Different from the Hyperscale Cloud
6.1 Hyperscale Cloud Computing Addresses The Issues Of Economies Of Scale
6.1.1 Mega Data Center Scaling
6.1.2 Mega Data Center Automatic Rules and Push-Button Actions
6.1.3 Keep It Simple Principle

7. Amazon Capex for Cloud 2.0 Mega Data Centers
7.1 Amazon Capex Dedicated To Support Datacenter
7.2 AWS Server Scale
7.3 Amazon North America
7.4 Amazon North America List of Locations
7.5 Innovation a Core Effort for Amazon
7.6 Amazon Offers the Richest Services Set
7.6.1 AWS Server Scale
7.7 On AWS, Customers Architect Their Applications
7.8 AWS Scale to Address Network Bottleneck
7.9 Networking A Concern for AWS Solved by Scale
7.10 AWS Regions and Network Scale
7.11 AWS Datacenter Bandwidth
7.12 Amazon (AWS) Regional Data Center
7.12.1 Map of Amazon Web Service Global Infrastructure
7.12.2 Rows of Servers Inside an Amazon (AWS) Data Center
7.12.3 Amazon Capex for Mega Data Centers
7.13 Amazon Addresses Enterprise Cloud Market, Partnering With VMware
7.13.1 Making Individual Circuits And Devices Unimportant Is A Primary Aim Of Fabric Architecture

8. Clos Network Architecture Topology
8.1 Google Clos Network Architecture Topology Allows the Building a Non-Blocking Network Using Small Switches
You Have To Hit A Certain Scale Before Clos Networks Work
8.1.1 Clos Network
8.2 Digital Data Expanding Exponentially, Global IP Traffic Passes Zettabyte (1000 Exabytes) Threshold

9. Summary: Economies of Scale

List of Figures:

Figure 1. Slow Growth Companies Do Not Have Data Center Scale

Figure 2. Mega Data Center Fabric Implementation

Figure 3. Facebook Schematic Fabric-Optimized Datacenter Physical Topology

Figure 4. Facebook Automation of Mega Data Center Process

Figure 5. Facebook Altoona Positioning Of Global Infrastructure

Figure 6. FaceBook Equal Performance Paths Between Servers

Figure 7. FaceBook Data Center Fabric Depends on Scale

Figure 8. Facebook Fabric Operates Inside the Data Center, Fabric Is The Whole Data Center

Figure 9. Fabric Switches and Top of Rack Switches, Facebook Took a Disaggregated Approach

Figure 10. Exchange Of Data Between Servers Represents A Complex Automation Of Process

Figure 11. Samsung Galaxy J3

Figure 12. Facebook Back-End Service Tiers And Applications Account for Machine-To-Machine Traffic Growth

Figure 1. Facebook Data Center Fabric Network Topology

Figure 13. Implementing building-wide connectivity

Figure 14. Modular Design Allows Users To Quickly Scale Capacity In Any Dimension

Figure 15. Facebook Back-End Service Tiers And Applications Functions

Figure 16. Using Fabric to Scale Capacity

Figure 17. Facebook Fabric: Pod Unit of Network

Figure 18. Server Pods Permit An Architecture Able To Implement Uniform High-Performance Connectivity

Figure 19. Non-Blocking Network Architecture

Figure 20. Facebook Automation of Cloud 2.0 Mega Data Center Process

Figure 21. Facebook Creating a Modular Cloud 2.0 mega data center Solution

Figure 22. Facebook Cloud 2.0 mega data center Fabric High-Level Settings Components

Figure 23. Facebook Mega Data Center Fabric Unattended Mode

Figure 24. Facebook Data Center Auto Discovery Functions

Figure 25. Facebook Automated Process Rapid Deployment Architecture

Figure 26. Google Douglas County Cloud 2.0 Mega Data Center

Figure 27. Google Data Center Efficiency Measurements

Figure 28. Google Andromeda Cloud High-Level Architecture

Figure 29. Google Andromeda Software Defined Networking (SDN)-Based Substrate Functions

Figure 30. Google Compute Engine Load Balancing Functions

Figure 31. Google Compute Engine Load Balanced Requests Architecture

Figure 32. Google Compute Engine Load Balancing Scaling

Figure 33. Google Traffic Generated by Data Center Servers

Figure 34. Google Mega Data Center Multipathing: Implementing Lots And Lots Of Paths Between Each Source And Destination

Figure 35. Google Mega Data Center Multipathing: Routing Destinations

Figure 36. Google Builds Own Network Switches And Software

Figure 37. Google Clos Topology Network Capacity Scalability

Figure 38. Schematic fabric-optimized Facebook datacenter physical topology

Figure 39. Google Jupiter Network Delivers 1.3 Pb/Sec Of Aggregate Bisection Bandwidth Across A Datacenter

Figure 40. Microsoft Azure Cloud Software Stack Hyper-V hypervisor

Figure 41. Microsoft Azure Running Docker Containers

Figure 42. Microsoft Data Center, Dublin, 550,000 Sf

Figure 43. Microsoft-Azure-Stack-Block-Diagram

Figure 44. Microsoft Azure Stack Architecture

Figure 45. Microsoft Data Centers

Figure 46. Microsoft Open Hardware Design: Project Olympus

Figure 47. Microsoft Open Compute Closes That Loop On The Hardware Side

Figure 48. Microsoft Olympus Product

Figure 49. Microsoft Azure Has Scale

Figure 50. Mega Data Center Cloud vs. Hyperscale Cloud

Figure 51. Amazon Web Services

Figure 52. Amazon North America Map

Figure 53. Amazon North America List of Locations

Figure 54. Woods Hole Bottleneck: Google Addresses Network Bottleneck with Scale

Figure 55. Example of AWS Region

Figure 56. Example of AWS Availability Zone

Figure 57. Example of AWS Data Center

Figure 58. AWS Network Latency and Variability

Figure 59. Amazon (AWS) Regional Data Center

Figure 60. A Map of Amazon Web Service Global Infrastructure

Figure 61. Rows of Servers Inside an Amazon (AWS) Data Center

Figure 62. Clos Network

Figure 63. Data Center Technology Shifting

Figure 64. Data Center Technology Shift

Amazon

Facebook

Google

Microsoft

Samsung

This website uses cookies to ensure you get the best experience. Learn more