v2.0 Series release notes

Included here are the release notes for all releases in the v2.0 release series.

v2.0.11

Released January 23, 2020.

Downloads

Binaries

macOS       Linux

Docker

YSQL Changes

• [YSQL] Colocation: Fix txn conflict issues due to absence of cotable ID in DocKey. #3251
• [YSQL] Table colocation drop flow. #3035
• [YSQL] Make parentheses optional for SPLIT clause. #3308
• [YSQL] Fixed restarted portal fields having wrong memory context. #3377
• [YSQL] IN clause pushdown for hash columns. #3130
• [YSQL] Reorganized and fixed INDEX implementation. #2959
• Access pg_stat_statements contents at <tserver_ip>:13000/statements. #3091
• [colocation] Create table tombstone. #3357
• [YSQL] Fix memory leak for altering temporary table. #3031

YCQL Changes

• [YCQL] Fixed Upgrade issue for SELECT from Indexed Table. #3281

System improvements

• Fix race condition on RetryingTSRpcTask::UnregisterAsyncTask. #3277
• [2DC] Change cdc_state table format. #3050
• Fix clock update during commit with committing transaction. #2571
• Don't check doc hybrid time at end of the key during rocksdb seek. #3024
• Fix redis keys command. #3311
• Move snapshot related classes from ent/ folder. #1032
• Use correct string end for char[] arg in Format function. #3396
• Fix SIGSEGV due to null consensus pointer in GetChanges. #3289
• Do not use TOMBSTONED tablets when load balancing across disks on a node. #3374
• Expose percentile metrics from histograms. #3362

Platform

• [YW] UI change for RBAC update to make Profile section functional
• [YW] Refreshing on the Edit Universe page does not load AZSelectorTable #3291
• Minor fix to ignore instances without memory specifications. #3298
• [YW] Fix missing property in newly generated cluster deviceInfo in ClusterFields. #3290
• Fix edit universe bug for region/zone update. #3323
• Cron expression validator on UI doesn't work correctly #3160. #3160
• AWS Instance Type Parsing Fix. #3063
• Added keyspace to multi-table backups. #3343
• Flow assign_public_ip through properly when creating GCP instance. #3386
• Added Audit trail for users. #3370

Note on upgrades from pre 2.0 releases

There are two ways to address the above note.

• If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
• If you want to continue using your existing data, then please save your data, delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

v2.0.10

Released January 6, 2020.

Downloads

Binaries

macOS       Linux

Docker

YSQL Changes

• [YSQL] Fixed an issue with PgDocOp lifespan sometimes being too short.
• [YSQL] "UPDATE" fails if any "NOT NULL" columns aren't updated #3174
• [YSQL] Force network library initialization on postmaster startup on macOS. 2509

YCQL Changes

• [YCQL] Table stuck in the keyspace after deletion #3032

System improvements

• Implement one way bitmap and Extract CleanupTransaction from YBTransaction. #3220
• Improved rocksdb corruption error logging. #2961
• Added TCMalloc stats when dumping memory usage. #3190

Platform

[YW] Fix infinitely updating state causing React to fail. Reason is calling setState repeatedly in componentDidUpdate will create an infinite depth nested state update.

Note on upgrades from pre 2.0 releases

There are two ways to address the above note.

• If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
• If you want to continue using your existing data, then please save your data, delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

v2.0.9

Released December 13, 2019.

Downloads

Binaries

macOS       Linux

Docker

YSQL Changes

• [YSQL] Push down LIMIT clause for INDEX_ONLY_SCAN. #3128
• [YSQL] Drop DB persist, delete sys tables. #2997
• [YSQL] Correctly report unsupported features in plpgsql grammar. #2005
• [YSQL] Enable DISCARD statement in grammar. #2491
• [YSQL] Update only the assigned columns in UPDATE statement. #3106
• [YSQL] Move YBCPgSession and PgTxnManager into PgApiImpl. #3053
• [YSQL] Deprecate num_tablets in CreateTableRequestPB
• [YSQL] Support create colocated db/table . #3033

YCQL Changes

• [YCQL] Support conditional DMLs in transaction statement block. #166
• [YCQL] Compatibility Issue - Not processing older format of protobuf message when picking index. #3175

System improvements

• Enable CXX11 ABI. #3116
• Limited transactional DDL: each DDL statement in its own transaction. #3108
• Improved TcpStream::sending_ queue memory tracking. #3041
• Per service gauge counter for number of enqueued rpcs. #2389
• Handle GetChanges(MinimumOpID) as a Request for the First Log Entry. #3036
• Fixed vmodule logging configuration. #3176
• Log over-replicated tablets in load balancer. #3173
• Fix for stuck Inbound RPC connections due to hitting memory limit. #2562
• Retry transaction status table creation if it fails. #3182
• Add ability to set transaction priority range using SQL. #3024
• Delete provisional records RocksDB SSTable files. #2186

Platform

• [YW] Change Intro dialog to only appear on login. Add checkbox in Intro dialog for permanently hiding modal. #3138
• [Platform] Fix console errors in UI code #2790
• [YW] Fix backup Universe from changing to undefined url
• [Platform] Task history not reflecting when a encryption is enabled/disabled/rotated #2929
• Add per node metrics for health. #2894
• Implement RBAC in YugaWare. #2793
• Add chrony configuration while provision step. #3038

Note on upgrades from pre 2.0 releases

There are two ways to address the above note.

• If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
• If you want to continue using your existing data, then please save your data, delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

v2.0.8

Released December 13, 2019.

Downloads

Binaries

macOS       Linux

Docker

YSQL Changes

• [YSQL] Enable client-to-server TLS encryption for postgres process when use_client_to_server_encryption is set to true. #2980
• [YSQL] Disable the unnecessary pggate-level auto read restarts. #2812
• [YSQL] yb-admin command to delete namespace, table, and index. #3069
• [YSQL] Assertion failed on TRUNCATE TABLE statement. #1632
• [YSQL] Consider HASH column when seeking index for ORDER BY clause. #2792

YCQL Changes

• [YCQL] Fixed 'kong migrations bootstrap' failure with yugabyte. #2829

System improvements

• Add log retention by op id policy documentation. #2773
• Switch Linuxbrew to 20181203T161736. #3044
• Check transaction status after load. #2186
• [2DC] Make cdc_poller log messages more descriptive
• Lower max timeout on master async tasks from 1 hour to 15 minutes. #1655
• Clean up intents written for foreign key check. #3111

Platform

• [YW] Add a Prometheus metrics export endpoint. #2459
• [YW] Recursively call task status API sequentially so there is no overlap. #2998
• [Platform] Move the CMK policy to be moved to KMS instead of having it universe level #2942

Note on upgrades from pre 2.0 releases

There are two ways to address the above note.

• If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
• If you want to continue using your existing data, then please save your data, delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

v2.0.7

Released December 04, 2019.

Downloads

Binaries

macOS       Linux

Docker

System improvements

• Expose rocksdb max_write_buffer_number as a gflag. #2935
• Roll forward master tablet deletion on fault during DeleteTable. #2721
• Batch Records on 2DC Consumer. #2580
• Use raft commit hybrid time for write batch frontier. #2912
• Remove aborted transaction from follower. #2186
• Add leader_stepdown to yb-admin.#1943
• [2DC] TLS support for 2DC
• CDC op id retention policy. #2486
• Fix double cleanup of provisional records. #2186
• Improved RPC heartbeats sending. #2964

Platform

• [YW] Fix node metrics on Universe specific Nodes page. #3006
• Fix the host info query to be called only on provider config page. #1342

Note on upgrades from pre 2.0 releases

There are two ways to address the above note.

• If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
• If you want to continue using your existing data, then please save your data, delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

v2.0.6

Released November 23, 2019.

Downloads

Binaries

macOS       Linux

Docker

YSQL changes

• [YSQL] Transparent read restarts for general queries. #1237
• [YSQL] Support row level security (RLS) as per Postgres. #1870
• [YSQL] use database type in catalog_manager for keyspace name uniqueness. #1476
• [YSQL] Enable fmgr cache for C functions. #2596
• [YSQL] fix delete from table returning * query with dropped columns. #2398
• [YSQL] Update catalog manager data while renaming YSQL table, DB and columns. #2280
• [YSQL] FOR [NO KEY] UPDATE. #2842
• Use transaction pool for YSQL. #2909
• [YSQL] Pick transaction id such that its leader is local to transaction coordinator. #2889
• [YSQL] Increase default PostgreSQL output buffer size; fix a bug with unnamed_stmt_psrc. #2975
• [YSQL] use ~/.yugabytedb as default certificate folder for ysqlsh #2985

System improvements

• [docdb] Speedup data loading in Master Leader startup. #2663
• [docdb] Speedup CREATE DATABASE for YSQL.
• Set Timeout and elapsed time in same units #2826
• Track memory taken by CDC producer. #2888
• Cleanup TIMED_OUT calls from /rpcz. #2807
• Improve logging of shutting down idle and broken connections. #2918
• Do not display user index tables as system tables in master UI. #2921
• CDC Producer Metrics. #2154
• yb-admin support for in-memory encryption keys. #2916
• Poll TCMalloc stats on update of root MemTracker. #2566
• Fix LB enable / disable handling. #1942
• Propagate hybrid time from follower. #2956

Platform

• [Platform] - Rotate key with service should not rely on alias/cmk already existing #2943
• [YW] Enable alerts email button is not clickable #2827
• [Platform] - Fix SetUniverseKey task for unencrypted universes #2976
• Fix k8s metrics for CPU and Disk Usage.
• [Platform] Support instance profiles for default CMK policy #2949

Note on upgrades from pre 2.0 releases

There are two ways to address the above note.

• If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
• If you want to continue using your existing data, then please save your data, delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

v2.0.5

Released November 15, 2019.

Downloads

Binaries

macOS       Linux

Docker

YSQL changes

• [YSQL] Change HINT text for unsupported "alter procedure/function". #2718
• [YSQL] Support client-server TLS encryption #2471
• [YSQL] Fix Master crash when deleting YSQL database #2665
• [YSQL] Fix read point for row lock stmt. #2523
• [YSQL] Transparent read restarts for aggregate queries. #2216
• [YSQL] Show oid for YSQL tables in master UI. #1525

YCQL changes

• [YCQL] Server crashes on WHERE condition for key_column and collection_column. #2638
• [YCQL] NULL value for clustering PK column causes driver to keep retrying. #2737
• [YCQL] Support CREATE TABLE ... WITH NUM TABLETS. #2448
• [YCQL] Support nested user-defined types #2729

System improvements

• Pipe cmk policy into EncryptionAtRestService #2714
• [CDC] Include CDC consumer op ID while determining op ID to evict from cache. #2317
• [docdb] Roll forward master tablet deletion on fault during DeleteTable #2721
• Add optional placement_uuid to yb-admin. #2589
• Truncate fails after add_node, problem on bootstraping #2804
• Update glog to fix issue with stack trace symbolization. #2800
• Implement new write throttling algorithm based on number of SST files in leader and followers. #2750
• Improve sending CQL result from TServer. #1450
• Fix non transactional conflict resolution. #2823
• Do not flush rocksdb memtable when user drops table. #2719
• Add and update placement ID for read replicas. #2876
• [CDC] Use new GetChanges API on CDC Consumer. #2863
• Fix DeleteTable concurrency issues. #2799
• [2DC] Don't log CDC record data. #2841
• Add support for in memory universe keys #2682
• Lower consensus_max_batch_size_bytes default to 4MB. #2739
• Inbound RPC calls are not destroyed until timeout or service shutdown #2887
• Don't reset cdc_state checkpoint to 0.0. #2897
• Change failure handling in ProcessRpcStatus. #2879
• Do smart consumer producer tablet mapping. #2679
• Add Java Client Support for Encryption at Rest #2906
• Generate new rejection score on retry. #2750

Platform

• [Platform] Add ability to turn off encryption at rest for an encrypted at rest universe #2730
• [Platform] Cache universe key registry in memory on YW side #2770
• [Platform] Add ability to launch AWS instances with encryption enabled storage #2759
• [Platform] Scheduled backups for Single table and Full universe flows.
• [Platform] Use a YW default CMK policy instead of the AWS default policy #2896
• [Platform] Enable support for Rotate Key/Disable Key/Enable Key flows. #2927
• [Platform] Integrate YW KMS with in-memory encryption at rest #2915
• [Platform] Rotate key with service should not rely on alias/cmk already existing #2943
• [Platform] Periodic job to set latest universe key into master from YW #2728

Note on upgrades from pre 2.0 releases

There are two ways to address the above note.

• If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
• If you want to continue using your existing data, then please save your data, delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

v2.0.3

Released October 28, 2019.

Downloads

Binaries

macOS       Linux

Docker

YSQL changes

• [YSQL] Set enable_ysql to true by default for YB-Master and YB-TServer. #2455
• [YSQL] Build YBTupleID value with respect to DocDB column order. #2438
• [YSQL] Support TLS Server-Server Encryption through YSQL API. #1845
• [YSQL] Check constraints when doing UPDATE. #2361
• [YSQL] Add default password to yugabyte user for ysqlsh. #2594
• [YSQL] authentication and authorization. #2610
• [YSQL] client authentication update. #2614
• [YSQL] Enable FOR SHARE and FOR KEY SHARE row locking in YSQL SELECT statements. #1199
• [YSQL] Avoid namespace id resolving for yb-admin import_namespace command. #2582
• [YSQL] SERIALIZABLE READ ONLY DEFERRABLE. #2161

YCQL changes

• [YCQL] Fix for SIGBUS TS crash on invalid statement. #2476
• [YCQL] A role shouldn't be able to modify the superuser status of itself or of a role granted to it. #2505
• [YCQL] Fix issue with permissions inheritance. #2550
• [YCQL] Mangling the column name in IndexTable for index selecting processing. #741
• [YCQL] Wrong port sent for NEW_NODE topology change events. #2607
• [YCQL] Fixed: Cannot CREATE TYPE when user has CREATE permissions for keyspace. #2556
• [YCQL] Add support for IF clause in SELECT statement. #2640
• [YCQL] Add description for JSONB INDEX and IF clause in SELECT. #2651
• [YCQL] Not reserving keyword USER and WHEN. #1661
• [YCQL] EXPLAIN SELECT causes tserver to crash. #2616

System improvements

• Added missing extractor checks into version_edit, so ldb doesn't crash. #2510
• Fix race in Batcher. #2501
• [2DC] Update checkpoint for replicated records, ignore tablet status while looking up peer. #2487
• Reduce log level while skipping over display of stats for tablets with state RaftGroupStatePB::SHUTDOWN. #1488
• [2DC] Send from_checkpoint only when consumer knows the last checkpoint. #2522
• Remove may exist state from YBTransaction. #2501
• [CDC] Setup Java Console Connector for CDC. #2081
• Fix operation memory double tracking. #1436
• Reinit total replicas to move when master leader fails during tserver blacklisting. #2453
• Display num sst files for tablets in yb_admin list_tablets_for_tablet_server. #2536
• Use boost::amotic for op_id_copy_ member (128 bits structure). #2545
• Derive proxy rpc addresses. #1023
• Update consensus meta term during local bootstrap. #2570
• Add metrics for thread count in each category. #1785
• Cleanup queue on timeout. #2592
• Do not call LogReader::Open() twice when skip_wal_rewrite is enabled. #2124
• Display num log segments for tablets in yb_admin list_tablets_for_tablet_server. #2555
• Fix Bug in YEDIS Keys Command. #2381
• Fix num tablets per table. #2630
• modify_placement_info clears read replica config. #2628
• Fix use of load balancer concurrent action limits. #2624
• Node should be able to transition from master to shell mode and back to master. #2007
• Use new linuxbrew (20191015T202549). #2608
• Remove in_memory_erase option and use SingleDelete instead of Delete. #2569
• Fixed invalid read-time for RF==1 case. #2394
• Move Rpcs to CDC consumer from CDC poller. #2549
• [CDC] Make redirection to tablet leaders for GetChanges asynchronous. #2503

Note on upgrades from pre 2.0 releases

There are two ways to address the above note.

• If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
• If you want to continue using your existing data, then please save your data, delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

v2.0.1

Released October 08, 2019.

Downloads

Binaries

macOS       Linux

Docker

YSQL changes

• [YSQL] Allow ALTER COLUMN add/set/drop identity. #2226
• [YSQL] PostgreSQL error code handling to avoid string-based error comparisons. #1289
• [YSQL] Fix duplicate key error being suppressed in COPY. #2289
• [YSQL] VERIFY_RESULT creates undesirable copy of Result<T&>'s value. #2298
• [YSQL] Warn on non-txn COPY with incompatible relation. #2278
• [YSQL] PostgreSQL error code handling to avoid string-based error comparisons. #1289
• [YSQL] Aggregate pushdowns in YSQL. #1851
• [YSQL] Skip dropped columns when iterating through the columns for UPDATE statement. #1969
• [YSQL] Use keyspace id instead of name in requests from yb-admin to catalog_manager. #1476
• [YSQL] Fix insert into temp table in function body assertion error. #1541
• [YSQL] Fix memory leak in psql. #2388
• [YSQL] Add default password for yugabyte user. #2465

YCQL changes

• [YCQL] Fixed permission check for DROP TYPE. #2373
• [YCQL] Fix for 'value too long' error code mapping. #2116
• [YCQL] 'expression not yet supported' error for old-format index update. #2470

System improvements

• Do not retry flush/compact async tasks. #2229
• Make yb-admin log an error message instead of fatal-ing on failed init. #2141
• Propagate timeout to transaction. #2334
• Warn on non-txn COPY with incompatible relation. #2278
• [2DC] Add Exponential Backoff to CDC Consumer. #2362
• Add support for cotable id in EnumerateIntents (needed for transactional DDL). #1404
• TServer blacklisting for leadership role. #1748
• Avoid unnecessary RocksDB lookup when we have multiple ops to one tablet. #2186
• [2DC] Reduce checkpoint frequency, avoid self looping in CDC producer, apply polling delay correctly. #2364
• Request intents cleanup even on failed replicas. #2411
• Add number of sst files to tables and tablets pages for tserver. #2230
• Fix role and term cache. #2267
• yb-admin add RF to add_read_replicas. #2395
• [2DC] Fix transaction status and record handling in CDC producer/consumer. #2386
• Handle concurrent shutdown and remote bootstrap. #2428
• [CDC] Fix bug in condition check for tablet leader. #2449
• Fix remote bootstrap for encryption at rest. #2462
• Implement dynamic priorities for compaction tasks. #2004
• [2DC] Update checkpoint for replicated records, ignore tablet status while looking up peer. #2487
• [2DC] yb-admin command to pause/resume replication. #2367

Note on upgrades from pre 2.0 releases

There are two ways to address the above note.

• If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
• If you want to continue using your existing data, then please save your data, delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

v2.0.0

Released September 17, 2019.

Downloads

Binaries

macOS       Linux

Docker

Release highlight

YSQL is now generally available and is ready for production deployments. Details available in Announcing YugabyteDB 2.0 GA: Jepsen Tested, High-Performance Distributed SQL.

File format change

There are two ways to address the above note.

• If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
• If you want to continue using your existing data, then please save your data, delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

YSQL changes

• [YSQL] Add CREATE TABLE tablet splits to grammar. #1486
• [YSQL] Allow ALTER TABLE ALTER COLUMN SET|DROP NOT NULL. #1200
• [YSQL] Add retry logic to pgbench. #2058
• [YSQL] Optimize single-row UPDATE and DELETE statements. #1028
• [YSQL] IN predicate pushdown. #1554
• [YSQL] Add top-level YSQL configuration gflag settings. #1140
• [YSQL] Add support for various ALTER TABLE ADD COLUMN variants. #2055
• [YSQL] Support UDTs and records. #1676
• [YSQL] Order range columns with respect to primary key description. #1250
• [YSQL] Support ALTER TABLE ADD COLUMN with default value. #2055
• [YSQL] Enable ANALYZE, VACUUM in grammar. #2217
• [YSQL] Non-transactional COPY in YSQL. #1926
• [YSQL] CREATE AGGREGATE, DROP AGGREGATE, CAST, OPERATOR, OPERATOR CLASS. #1981
• [YSQL] Create yugabyte user and yugabyte as default user and database. #2277

YCQL changes

• [YCQL] Fixed TS crash if a built-in function argument is a column. #1944

System improvements

• Propagate intents apply failure to client. #1974
• Check shutdown state while scheduling compaction. #2100
• Add leader check on master TS Write interface. #1995
• Correctly handle duplicate key in postgres layer. #2012
• Remove StopWatch from RocksDB Seek. #2136
• Avoid picking read time for the serializable transactions. #2125
• CDC: Producer and consumer master handshake to set up 2DC. #1977
• Optimized Drop Table Locking. #2056
• Reduce number of sys_catalog writes: skip no change heartbeats. #1966
• CDC: Consumer changes. #1481
• CDC: Add master-master replication support. #1824
• Increase WAL retention time when CDC stream is setup on table. #2050
• Apply changes received from producer universe. #2153
• CDC: Delete for DROP DATABASE. #2114
• Show Load Balancer state in Master UI. #646
• Add support for error code categories to Status. #2200
• [2DC] Handle RPC controller errors and CDC consumer shutdown. #2231
• Don't apply transactions on tablets without intents. #2070
• Allow dynamic priorities in PriorityThreadPool. #2004
• Added rocksdb_universal_compaction_always_include_size_threshold. #2085
• Do full table name match while finding matching tables in 2DC. #2250
• Fix cleaning Raft log after transactional writes are stopped. #2221
• Fix handling of calls expired in queue. #2269
• Load balancer should not be considered idle when tablet remote bootstrap. #2193
• [YSQL]: Fix number of tablets created for system tables. #2293
• Reduce fan-out and use tablet invoker for writing data.
• SerializableReadWriteConflict needs better success probability.