Vitastor

Hierarchical failure domains, handling of 100% full OSDs, nbd-netlink and more.

After half a year of hard work, VitastorFS is finally here ! :-)

Do not use \r if output is not a terminal (should fix unexpected job output in proxmox)
Fix rm/rm-data error return code, add --down-ok option to bypass the error
Add EIO retry timeout and allow to disable these retries, rename up_wait_retry_interval to client_retry_interval
Add ubuntu jammy build
Wait for blockstore initialisation before starting OSD (prevent timeouts when init takes time)
Fix a rare use-after-free in automatic sync after delete in blockstore

Fix another old “BUG: Attempt to overwrite used offset” in a very simple case: bs=4k rw=write iodepth=16 from OSD start; add this case to tests
Fix a rare crash with “unexpected state during flush: 0x51” possible with EC since 1.4.2 during rebalance and OSD outages
Fix a rare write stall with EC & immediate_commit=none caused by sync operations reserving unneeded space in the journal
Fix 32-bit build warnings, most in printf/scanf format strings

Unwavering stabilization of 1.4.x, continued :-)

Include the accidentally lost part of 1.4.5 journal trimming fix
Fix a possible OSD crash with “BUG: Attempt to overwrite used offset” which was probably present for long time, but became apparent after fixing flapping tests in CI
Fix remaining flapping tests in CI. It was the first time when tests actually passed without retries :-)

Fix a write stall caused by incorrect journal trimming introduced in 1.4.4 :)
Fix PGs sometimes hanging in “starting” state on mass OSD restarts
Fix a rare crash with “map::at” during OSD pings
Use new defaults for non-capacitor (desktop) SSDs - improves T1Q256 random write from ~6k iops to ~45k iops
Make journal_trim_interval configurable

A couple of fixes for EC pools

Fix a segfault possible on partial EC overwrite in 1234 -> 5030 rebalance scenario
Fix two problems leading to EC pools stalling on rebalance & parallel sudden stops of OSDs, for example during a sudden poweroff of a host:
- Recovery auto-tuning (1.4.0 feature) could apply too large delays and stall the EC journal - fixed by limiting delays with a new recovery_tune_sleep_cutoff_us parameter (10 seconds by default) and applying recovery pauses before write operations, not after them, to not occupy space in the journal for long time
- Dynamic journal space reservation (1.3.0 feature) wasn’t accounting new writes when checking the limit so OSDs could still fill the journal fully and stall - fixed by including new writes into the limit
Print etcd dbSize instead of dbSizeInUse in status

Hotfix for hotfix O:-)

“Write stall fix” was incomplete and EC write stalls could continue even on 1.4.2. Now they’re finally fixed O:-)
Make monitor ignore statistics of stopped OSDs. Previously if you stopped all OSDs the last total I/O numbers would remain the same indefinitely

Log to systemd by default
Fix excessive autosyncs after every operation with disabled immediate_commit (introduced in 1.1.0)
Fix a possible write stall with EC due to the lack of OSD wakeup after stabilizing previous writes
Change sync operation semantics as a final fix to possible write stalls with EC and disabled immediate_commit
Sync after deleting data in CLI rm / rm-data if immediate_commit is disabled
Fix OSDs ignoring syncs & autosyncs for delete operations
Fix OSD space reporting sometimes adding garbage zeros for deleted inodes (causing extra pool/stats etcd keys for deleted pools)
Speed up monitor failover - change default etcd_mon_ttl from 30 to 5 seconds
Speed up operation retries - change default up_wait_retry_interval to 50 ms
Add patch for libvirt 9.10

Fix a monitor crash on primary OSD switching introduced in 1.4.0
Fix “partly outside array bounds” warnings for GCC 12 in cpp-btree
Fix a realloc memory leak in theory possible with too large listings (OSD_OP_LIST)

Posts up to 2024-04-09