Chapter 11. File Locking

11.1.1. Exclusive and shared locks

Both fcntl and flock give the choice of either an exclusive lock, where only one process could hold the lock, or a shared lock, where multiple holders could simultaneously exist, to the exclusion of holders of the exclusive lock. The exclusive lock is sometimes called a "single writer" lock, because its exclusive nature lends itself to allowing safe writes to a file. The shared lock is sometimes called a "multiple readers" lock because its shared nature lends itself to allowing multiple safe reads of a file.

11.1.2. Record locks

The fcntl system call also has the feature of byte range record locking. This means that the application can partition a file into as many arbitrarily sized segments or records that it wants, and by specifying a file offset and length, lock them. Thus, it is possible to have both an exclusive lock and a shared lock on a file, provided the file offsets and lengths of each record lock do not overlap.

11.1.3. Mandatory versus advisory locking

Both fcntl and flock offer advisory locking. Advisory locking is locking that requires the cooperation of participating processes. Suppose process A acquires an exclusive lock on the file, with the intent to write it. Suppose process B opens the file with the intent to write it. If process B fails to acquire a lock, there is nothing to prevent it from issuing a write system call and corrupting the process that A is writing. For this reason, advisory locking is sometimes called unenforced locking.

System V (and therefore Solaris) offers mandatory or enforced locking as an option. This option is enabled if mandatory lock permissions are set on a file. Mandatory lock permissions are an overload of the set group ID execution bit (02000 in octal). If the set group ID execution bit is set, and if the group execution bit is not set, then all reads and writes to the file will use enforced locking. So, for example:

% chmod 2644 example
% ls -l example
-rw-r-lr--   1 mre      staff            9 Dec 28 10:52 example

This makes file example readable and writable by the file's owner, and readable by everyone else. The appearance of the l in the first field of the output of the ls command tells you that mandatory locking is enabled. Of course, you can use any combination of read or write permissions for the file's owner, group, and world.

If the mandatory lock permissions are set on a file, then every write( ) or read( ) system call results in an implicit sequence of:

fcntl(...); /* lock the file at the range we are reading or writing */
read(...); /* or */ write(...);
fcntl(...); /* unlock the file at the range locked above */

What if the process has already acquired a lock by an explicit fcntl call? If the range locked is equal to or encompasses the range the read or write is done on, then no implicit pair of fcntl calls are done. If the range explicitly locked partly overlaps the range read or write will do, then implicit fcntl calls are done on the unlocked portion of the range.

Mandatory locking seems very useful, but it is open to denial of service attacks. Suppose mandatory lock permissions are set on a file. An attacker named Mallet decides to issue an fcntl call to get an exclusive lock on the entire file. Bob now tries to read the file and finds that his application hangs. A proponent of mandatory locking might point out that the mistake was in allowing the file to be accessible by Mallet (if Mallet can't open the file, he can't lock it). The counter argument is that if you are going to rely on permissions to avoid a denial of service (and restricted permissions are a good thing to have for critical applications), then the set of users who can access the file is limited to those with a vested interest in avoiding denial of service. In that case, mandatory locking is no more useful than advisory locking.

11.1.4. Windows/NT locking scheme

The discussion so far has been about Unix locking paradigms. The Windows world has a different paradigm. There are two major differences between Unix and Windows locking:

• The first difference is that the Windows world supports a share reservation programming interface. Share reservations apply to the entire file and are specified at the time a file is created or opened. A share reservation consists of a pair of modes. The first is the access mode, which is how the application will access the file: read, write, or read/write. The second is the access that the application will deny to other applications: none, read, write, or read/write. When the application attempts to open a file, the operating system checks to see if there are any other open requests on the file. If so, it first compares the application's access mode with the deny mode of the other openers. If there is a match, then the open is denied. If not, then the operating system compares the application's deny mode with the access mode of the other openers. Again, if there is a match, the open is denied.

• The second difference is that there is no advisory locking. Whole file locking, byte range locking, and share reservation locking are all mandatory or enforced.[18]

  [18]As it turns out, very few Windows programs rely on byte range mandatory locking.

TIP: Share reservations in the Windows world do not interact at all with Windows byte range or whole file locking.