Gürsel Kaynak Senior Specialist Systems Engineer EMC Corporation
Disruptive / Opportunistic IT Trends Mobile Cloud Big Data Social T R U S T
BILLIONS OF USERS MILLIONS OF APPS Mobile Cloud Big Data Social Mobile Devices HUNDREDS OF MILLIONS OF USERS TENS OF THOUSANDS OF APPS LAN/Internet PC Client/Server MILLIONS OF USERS THOUSANDS OF APPS Source: IDC, 2013 Mainframe, Mini Computer Terminals
ADVANCED SECURITY : A Unique Federation Of Businesses Focused & Free To Execute On Their Mission, Strategically Aligned PLATFORM FOR CLOUD, BIG/FAST DATA APPS SOFTWARE-DEFINED DATA CENTER INFORMATION INFRASTRUCTURE
Our Dual Innovation Strategy Research & Development 12% Investment Technology Acquisitions String Of Pearls 10% Investment
The Business Drivers Increase Revenue Lower Operational Costs Reduce Risk
Cloud Transforms IT Infrastructure 1 Standardize 2 Virtualize 3 Automate
Two Distinct Architectures Next Gen Cloud Applications Traditional Applications 2016 48M SIMPLE DATA SERVICES 2012 6M 700% 2016 141M 2012 83M 70% ADVANCED DATA SERVICES HDFS / Object Storage Eventual Consistency Software-Based Resiliency Block / File Storage Transactional Consistency Hardware-Based Resiliency
Location & Types Of Big Data Enterprise Partner Public Unstructured Data Credit Data Shipping Data Structured Data Forecast Data Location Data Social, Video Data
Location & Types Of Big Data (& Fast!) Data Enterprise Partner Public Unstructured Data Credit Data Shipping Data Structured Data 10101010100101010 011001010101110010 1101010100101011111 Telemetry Data Forecast Data Location Data Social, Video Data
The Value Of Data Is Also Growing New Business Models Are Based On Data! 1,000,000,000 Queries / Day 250,000,000 New Photos / Day 290,000,000 Updates / Day 4,000,000 Claims / Day 2,800,000,000 Trades / Day 31,000,000,000 Interactions / Day
New Apps Are Built In New Ways Social & Primarily Accessed By Mobile Devices Developed With Modern Programming Tools & Data Fabrics Deployed On A Software-Defined Cloud Infrastructure Public Cloud
Introducing Pivotal PLATFORM FOR NEXT GENERATION CLOUD & BIG / FAST DATA APPLICATIONS 1250 EMPLOYEES ~$300M IN 2013 REVENUE
Open Architecture Provides Choice Private, Virtual Private & Public Cloud Deployment Options Next Generation Cloud & Big / Fast Data Apps Private Cloud Virtual Private Cloud Public Cloud
How Can Big Data Transform Your Business? New Source of Customer, Product and Operational Insights Today s Decision-making Big Data Decision-making Rearview Mirror hindsight Less than 10% of available data Incomplete, disjointed, inaccurate Forward-looking insight Exploit all data from diverse sources Real-time, correlated
Leveraging Full Power of the Family
Hadoop Adoption in the Industry 2007 2008 2009 2010 The Datagraph Blog Source: Hadoop Summit Presentations
Target Markets, Verticals & Pain Hadoop Use Cases by Vertical Finance Web 2.0 Telecom Healthcare Risk Modeling/Management Portfolio Analysis Investment Predictions Fraud Detection Compliance Check Customer Profiling Social Media Analytics ETL Network analysis based on transactions Product Recommendation Engine Search Engine Indexing (Search Assist) Content Optimization Advertising Optimization Customer Churn Analysis POS Transaction Analysis Data Warehousing Network Graph Analysis Call Detail Record (CDR) Analysis Network Optimization Service Optimization & Log Processing User Behavior Analysis Customer Churn Prediction Machine-generated data centralization (logs from firewalls, towers, switches, servers, etc..) Electronic Medical Record Analysis Claims Fraud Detection Drug Safety Analysis Personalize Medicine Healthcare Service Optimization Drug Development Healthcare Information Exchange Medical Image Processing
HOW DO WE SPEED DEPLOYMENT
Three Paths To Cloud Infrastructure EMC Products Partner VSPEX VCE VBLOCK Best Of Breed Infrastructure Components Proven Infrastructure Converged Infrastructure Simpler, Faster, Lower TCO
EMC VSPEX SIMPLE. EFFICIENT. FLEXIBLE. PROVEN. BY EMC.
OVER 1500 PARTNERS
CUSTOMERS VSPEX: More Customers In Half The Time 3000+ Customers In 40+ Countries 3000 2500 NetApp 2000 1500 1000 500 0 6 12 18 24 30 36 Source: EMC And NetApp Earnings Announcements MONTHS SINCE FLEXPOD LAUNCH
HOW DO WE MANAGE BOTH ARCHITECTURES
EMC ISILON BIG DATA STORAGE
Scale Capacity EMC CONFIDENTIAL INTERNAL USE ONLY 38 Geleneksel Scale-Up Darboğazlar... Isilon Scale-Out Eşzamanlılığın gücü Controller Disk Shelves Kontrolcüler & diskler Master / Slave sistemler yada 2-yollu clusterlar Bir çok dosya sistemi / Volumelar Bir çok yönetim N-Storage noktası Adacıkları ENTERPRISE CLASS ISILON IQ NODES CPU, Cache, I/O, Disks (optional 2 nd switch) (Intra-cluster Infiniband) 144 Node a kadar, 10TB dan 20PB a Tekil Peer-to-Peer Cluster OneFS Dağıtık Dosya Sistemi Tek bir Storage Pool/Namespace Tek Noktadan N-Yollu Yönetim Simetrik Cluster
EMC CONFIDENTIAL INTERNAL USE ONLY 39 EMC Isilon Çekirdek İnovasyonu OneFS Scale-Out Operating System
Geleneksel Name Node Mimarisi Name Node Namenode un otomatik olarak tamiri mümkün değildir. İleride Hadoop mimarisine failover gelse bile, manual olarak recovery gerekir. When Name Node Namenode map is verinin lost nerede or damaged, saklandığının data bilgisine location information sahiptir no longer exists
Isilon HDFS Arayüzü Isilon Namenode ve Datanode arayüzlerini kendi üstünde destekler. Dosya Sistemi olarak OneFs kullanılır. Sadece HDFS istemcilerine Isilon un ismini yazmanız ile çalışır!
HDFS bir protokoldür! Isilon nodeları ( 6.5.5 den itibaren) Namenode ve datanode protokollerini konuşabiliyor. Böylelikle bu iki servis için ayrı bir sunucu ve depolama cihazı yatırımı yapılmasına gerek yok! Cluster içindeki her Isilon node u hem namenode hem de datanode olarak çalışıyor (isi_hdfs_d) Veri aynı NFS yada SMB de olduğu gibi tüm OneFS dosya sistemine eşit olarak yayılıyor.
Clustered NameNode Isilon ile Dağıtık (Clustered) Name Node yapısı Name Node MTTDL = 5,000 yıl Metadata tüm node lar üzerine yayılıyor. Tüm node lar arasında kesintisiz ve sağlıklı bir yapı vardır. Isilon ile hem verileriniz, hem de HDFS sisteminizin verileri emanet altında
Snapshot/Versiyon Kontrol 5 5 Öncesi Geleneksel HDFS de replikasyon yoktur Verilerin snapshot ı alınamaz Versiyon kontrolü yoktur İş sürekliliği mümkün değildir. Sonrası HDFS e snapshot özelliği gelir. Multi-threaded li, Çoklu- Node Scale-Out replikasyon RPO/RTO zamanı belli Geo-replikasyonlu Hadoop!
İşlem gücü Isilon ile büyüme Yüksek performans ve kapasite! Hadoop Depolama İşlem gücü hızı sabit İşlemci gücünü arttırırken kapasite de arttırılır. QoS belirlemek zordur İşlem gücü güncellemesi için tüm sistem değişmek zorunda! depolama Isilon Depolama kısmına bağlı kalmadan işlemci gücünü arttırmak mümkün. İşyüküne göre performans almak mümkün. Depolama sistemi değişmeyeceği için veri taşımasına gerek yok. İstenildiği gibi donanım arttırmak mümkün.
Protokol Desteği Sunucular Sunucular Sunucular Sunucular Sunucular Öncesi HDFS, Windows, Unix, Linux, Apple, yada başka işletim sistemlerine görünür değildir. Big Data sadece Big Data için kullanılır. Sonrası Çoklu Protokol desteği ile tüm protokollerden erişim mümkündür.! Big Data gerçek verinizdir ve sizindir!
EMC VIPR
EMC ViPR, Software-Defined Storage ViPR Data Services: The Architecture For Next Gen Cloud Apps EMC ViPR Platform ViPR Data Services ViPR Controller Provisioning Self-Service Reporting Automation VMAX VNX Isilon 3 rd Party Commodity
ViPR Future Direction More ViPR Data Services & Support For Commodity Hardware EMC ViPR Platform ViPR Data Services ViPR Controller Provisioning Self-Service Reporting Automation VMAX VNX Isilon 3 rd Party Commodity
Why Is This So Transformative? RUN CLOUD APPS UNCHANGED Atmos Isilon 3 rd Party VNX USE EXISTING ARRAYS FOR OBJECT LEVERAGE COMMODITY HARDWARE
Trusted IT Means Identifying & Repelling Threats Advanced Security Protecting Data Integrated Backup & Recovery Ensuring Availability Of Applications, Systems & Data Continuous Availability
Adaptive & Risk-Based Authentication User Action Analysis Engine Private Cloud Read Email Username & Password Download Sales Pipeline Additional Authentication Two-Factor Device Profile User Behavior Profile Fraud Network Public Cloud Access Bank Account Username & Password Out Of Band Transfer Funds Challenge Q Additional Authentication
Centralize Data Protection Storage Data Domain Is Protection Storage ViPR Data Services ViPR Controller Data Domain VMAX VNX Isilon 3 rd Party Commodity
Continuous Availability VPLEX & RecoverPoint Become Advanced Data Services ViPR Data Services ViPR Controller Data Domain VMAX VNX Isilon 3 rd Party Commodity
A Closer Look At The Public Cloud.
HOW DO WE PROVIDE SIMPLE WEB SCALE STORAGE FOR BOTH ARCHITECTURES
PROJECT NILE ELASTIC CLOUD STORAGE FOR YOUR PRIVATE CLOUD OR YOUR ONLINE SERVICE
PROJECT NILE THE WORLDS FIRST COMMERCIALLY AVAILABLE COMPLETE WEB-SCALE STORAGE I N F R A S T R U C T U R E F O R Y O U R D ATA C E N T E R Block File Object COMPLETE Elastic Self-Service Exabyte+ WEB-SCALE Service Provider Enterprise ANY DATA CENTER
ADVANCED SECURITY Has Built A Differentiated Stack Best Of Breed. Architected Horizontally, Not Vertically. Choice. PLATFORM FOR CLOUD, BIG/FAST DATA APPS SOFTWARE-DEFINED DATA CENTER INFORMATION INFRASTRUCTURE
Sorularınız?