OmniSciDB  cde582ebc3
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
heavyai_locks.h
Go to the documentation of this file.
1 /*
2  * Copyright (C) 2022 HEAVY.AI, Inc.
3  */
4 
6 //
7 // DistributedSharedMutex dmutex{"example.lockfile", [&](size_t version){
8 // /* Called when we lock the mutex, but only when we need to invalidate
9 // * any cached copies of the data protected by the lockfile. */
10 // }};
11 //
12 // std::unique_lock write_lock{dmutex}; // Standard read-write lock.
13 // std::shared_lock read_lock{dmutex}; // Standard read-only lock.
14 //
15 // Optionally, instead of passing in the callback, a Callbacks struct can be
16 // passed in, allowing a variety of powerful callbacks.
17 
19 //
20 // The DistributedSharedMutex uses a tiny lockfile containing a version number.
21 // That version number gets incremented every time a process takes an exclusive
22 // (write) lock. Each process keeps track of the last version number seen, making
23 // it easy to detect when the data protected by the lockfile has changed.
24 //
25 // Expected to be NFSv4 compatible and Lustre 2.13 compatible.
26 //
27 // As an optimization, DistributedSharedMutex holds on to read locks for a brief period
28 // of time after the lock is released by the user. Testing shows that this optimization
29 // makes locking on a shared network filesystem (ex: NFS) nearly as fast as a local
30 // filesystem. (See: maybeExtendLockDuration(), g_lockfile_lock_extension_milliseconds)
31 //
32 // The lockfile actually uses standard POSIX locking, which is likely to save us a
33 // large amount of development and maintenance. However, unfortunately, we're stuck
34 // on an old version of the Linux kernel (HeavyDB releases are built on CentOS 7
35 // from 2014 using Linux kernel 3.10 from 2013) which was just before support for
36 // OFD locks were added to the kernel.
37 //
38 // The lack of OFD locks means we had to implement per-thread reference counting
39 // and thread queuing ourselves, which is tricky and error-prone. Hopefully we can
40 // remove half of the code in DistributedSharedMutex after we upgrade our build
41 // process to more recent kernels.
42 //
43 // If POSIX lockfiles ever turn out to be undesirable, we could create an alternate
44 // version of DistributedSharedMutex using our own simple lock server. In fact,
45 // writing such a server probably would have been easier than dealing with the
46 // missing OFD locks, but in the long run, POSIX lockfiles are expected to give us
47 // a better product with less maintenance and easier configuration (no separate
48 // lock server we would need to install).
49 //
50 // See also about OFD locks:
51 // https://lwn.net/Articles/586904/
52 // https://kernelnewbies.org/Linux_3.15#New_file_locking_scheme:_open_file_description_locks
53 // https://en.wikipedia.org/wiki/Readers%E2%80%93writer_lock#Priority_policies
54 // https://rfc1149.net/blog/2011/01/07/the-third-readers-writers-problem/
55 
56 #pragma once
57 
58 #error "file only available with EE"