1. docker
  2. Commit ec79d0fc054a97b1d18adec794c6c30bfb11a488
Browse code

pkg/fileutils: GetTotalUsedFds(): use fast-path for Kernel 6.2 and up

Linux 6.2 and up (commit [f1f1f2569901ec5b9d425f2e91c09a0e320768f3][1])
provides a fast path for the number of open files for the process.

From the [Linux docs][2]:

> The number of open files for the process is stored in 'size' member of
> `stat()` output for /proc/<pid>/fd for fast access.

[1]: https://github.com/torvalds/linux/commit/f1f1f2569901ec5b9d425f2e91c09a0e320768f3
[2]: https://docs.kernel.org/filesystems/proc.html#proc-pid-fd-list-of-symlinks-to-open-files

This patch adds a fast-path for Kernels that support this, and falls back
to the slow path if the Size fields is zero.

Comparing on a Fedora 38 (kernel 6.2.9-300.fc38.x86_64):

Before/After:

go test -bench ^BenchmarkGetTotalUsedFds$ -run ^$ ./pkg/fileutils/
BenchmarkGetTotalUsedFds 57264 18595 ns/op 408 B/op 10 allocs/op
BenchmarkGetTotalUsedFds 370392 3271 ns/op 40 B/op 3 allocs/op

Note that the slow path has 1 more file-descriptor, due to the open
file-handle for /proc/<pid>/fd during the calculation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Sebastiaan van Stijn authored on 2023/06/30 07:46:46
Showing 1 changed files
pkg/fileutils/fileutils_linux.go
History View file @ ec79d0f
... ... 
@@ -7,12 +7,27 @@ import (
7 7 
 	"os"
8 8
9 9 
 	"github.com/containerd/containerd/log"
10 
+	"golang.org/x/sys/unix"
10 11 
 )
11 12
12 13 
 // GetTotalUsedFds Returns the number of used File Descriptors by
13 14 
 // reading it via /proc filesystem.
14 15 
 func GetTotalUsedFds() int {
15 16 
 	name := fmt.Sprintf("/proc/%d/fd", os.Getpid())
17 
+
18 
+	// Fast-path for Linux 6.2 (since [f1f1f2569901ec5b9d425f2e91c09a0e320768f3]).
19 
+	// From the [Linux docs]:
20 
+	//
21 
+	// "The number of open files for the process is stored in 'size' member of
22 
+	// stat() output for /proc/<pid>/fd for fast access."
23 
+	//
24 
+	// [Linux docs]: https://docs.kernel.org/filesystems/proc.html#proc-pid-fd-list-of-symlinks-to-open-files:
25 
+	// [f1f1f2569901ec5b9d425f2e91c09a0e320768f3]: https://github.com/torvalds/linux/commit/f1f1f2569901ec5b9d425f2e91c09a0e320768f3
26 
+	var stat unix.Stat_t
27 
+	if err := unix.Stat(name, &stat); err == nil && stat.Size > 0 {
28 
+		return int(stat.Size)
29 
+	}
30 
+
16 31 
 	f, err := os.Open(name)
17 32 
 	if err != nil {
18 33 
 		log.G(context.TODO()).WithError(err).Error("Error listing file descriptors")
... ... 
@@ -31,5 +46,7 @@ func GetTotalUsedFds() int {
31 31 
 			return -1
32 32 
 		}
33 33 
 	}
34 
+	// Note that the slow path has 1 more file-descriptor, due to the open
35 
+	// file-handle for /proc/<pid>/fd during the calculation.
34 36 
 	return fdCount
35 37 
 }