Günümüzde Postgres. Modern, Ölçeklenebilir Uygulamalar. Utku Azman Citus Data PGDay citusdata.com

Transkript

1 Günümüzde Postgres Modern, Ölçeklenebilir Uygulamalar Utku Azman Citus Data PGDay citusdata.com

2 Veritabanlarının basit olduğu zamanlar (2000 lere kadar) Open Source RDBMS Proprietary Analytics (OLAP) Workloads Operations (OLTP) 2

3 Bugün yapısal veritabanlarını zorlayan iki gelişme 1 Data with less structure 2 LOG Data Growth >> Silicon Growth Data 2x every 15 mo Moore s Law 2x every 24 mo 3

4 Postgres in geçmişinden Bu zorluklar yeni değil 1 SQL or not? (1995) Post-Ingres Started life as object store Added SQL API in Scaling out to handle data growth (2005) For analytics only: MPPs So many forks! AsterData, Netezza, ParAccel (Redshift), Greenplum 4

5 Bugün ise Postgres bu zorluklara karşı çok daha güçlü - SQL Extension APIS (2011) CREATE EXTENSION... Planner Executor Functions Custom scan Foreign tables Access methods Commit / abort PostgreSQL Extension (.so) 5

6 RDBMS ile günümüz ihtiyaçlarına cevap verme 1 2 Yapısal ve yapısal olmayan veri Ölçeklenme işleme & performans 6

7 Dosya sisteminden başlama (+) Any data, any structure (+) Infinitely scalable storage (+) Write fast (-) Pay cost at query time (-) Batch vs. real-time (-) Indexes (Append only FS) 7 (Hadoop)

8 Tek bir erişim şeklini destekleme Semi-structured (JSON) (+) Simple: Put & Get (+) Scalable (-) No expressiveness for analytics (-) No JOINS, data duplication (-) Enforce structure at app layer 8

9 1 Yapısal ve Yapısal olmayan veri Veritabanını JSON data tipiyle genişletme Table "public.events" Column Type Sample Data user_id bigint created_at timestamp :57: payload jsonb 9 Umur Cubukcu Strata Data Conference March 2018 The State of Postgres

10 2 Ölçeklenme İşleme ve performans İndex ve diğer temel performans araçları B-tree indexes GIN & GiST indexes Secondary indexes Full text search Index-only scans Fitting indexes into memory + Not to forget: Parallel queries, MVCC, and many more. 10 Umur Cubukcu Strata Data Conference March 2018 The State of Postgres

11 2 Ölçeklenme İşleme ve performans İşlem (ve join)leri birden fazla PostgreSQL instances a gönderme SELECT FROM (a JOIN b) SELECT FROM (a JOIN b) Data Node 1 Events_101 Events_103 Users_101 Users_103 SELECT FROM events a JOIN users b events users SELECT FROM (a JOIN b) SELECT FROM (a JOIN b) Data Node 2 Events_102 Events_104 Users_102 Users_ Data Node N 11

12 12 Postgres e yatay ölçeklenme için eklenti: Citus

13 PostgreSQL: Dinamik ve global ekosistem Sample PostgreSQL Extensions citus pgcrypto pg_cron pg_partman postgresql-hll cstore_fdw unaccent cube jdbc_fdw pg_trgm PostGIS pg_buffercache pg_prewarm btree_gin btree_gist postgis_topology pg_stat_statements postgresql-unit plpgsql plv8 pg_telemetry foreign data wrappers Integrations 13

14 PostgreSQL in artan momentumu Database adoption among developers 1 PostgreSQL MySQL MongoDB SQL Server + Oracle Source: % database job postings that mention each specific technology, across 20K+ job posts on Hacker News, 14

15 ve büyüyen kullanıcı kitlesi PostgreSQL popularity = Hadoop + Mongo combined Winning Startups & Enterprises PG Mongo Hadoop Source: Google Trends for the past 2 years 15

16 Postgres in uzun soluklu yürüyüşü Günümüz uygulamalarında Postgres stack in neresinde? 16

17 Modern iş yükleri de evrim geçiriyor Open Source Improvement workloads Application workloads RDBMS - Transactions - Short-requests - In-app analytics Proprietary Analytics (OLAP) Workloads Operations (OLTP) 17

18 Modern veritabanlarının desteklediği 3 tip uygulama Data volume 1 2 Systems of record Core workloads, transactions Real-time data Millisecond latencies Systems of engagement 3 Drive engagement & revenue Real-time data, multiple sources Sub-second latencies Systems of improvement Identify business process improvements Offline data, multiple sources Sub-minute / hour latencies, data analysts Application data Time to action 18

19 Altyapınızda PostgreSQL in yeri Application Data PostgreSQL NoSQL Standalone database Storage Compute Persistence layer for Kafka Kafka Spark HDFS / S3 Persistence layer for Spark Adjacent to NoSQL Note: Standard PostgreSQL connectors for all tools (e.g. ODBC / JDBC, PostgreSQL language bindings) available for integrations. 19

20 Özetle: PostgreSQL Eklentiler Esneklik Ekosistem 20

21 Ölçeklenme Video demo

22 github.com/citusdata/citus citusdata.com/jobs