File Descriptors

File descriptors are integer values that processes and the linux kernel use to manage open files. Any process that is spawned starts with 3 file descriptors: stdin(0), stdout(1), and stderr(2). In bash we can list the file descriptors of an empty process like this.

sleep 2 & ls -l /proc/$\!/fd
[1] 4473
total 0
lrwx------  0 -> /dev/pts/0
lrwx------  1 -> /dev/pts/0
lrwx------  2 -> /dev/pts/0

What happens when we open a new file? The process tells the kernel “I’d like to open a file”, and the kernel says “It’s now open, when you want to write to it, just refer to it by this descriptor value N”.

func main() {
	file, err := os.Open("example.txt")
	if err != nil {
		fmt.Println("Error opening file:", err)
		return
	}

	defer file.Close()

	fd := file.Fd()

	fmt.Printf("the file descriptor for 'example.txt' is: %d\n", fd)
}

One of the more famous uses of file descriptors is to redirect stderr output to stdin using the command 2>&1, or ignore it completely with 2 > /dev/null. The redirection operator is essentially telling the kernel “When I send you file descriptor 2, send it to the same location you have recorded for file descriptor 1”.

You can do these redirects in Go by using the Dup2 syscall. This example redirects stdout to a file.

func main() {
	file, err := os.OpenFile("example.txt", os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0o644)
	if err != nil {
		fmt.Println("failed to open file: ", err)
	}

	defer file.Close()

	fileFd := int(file.Fd())

	err = syscall.Dup2(fileFd, 1)
	if err != nil {
		panic(err)
	}

	fmt.Println("Hello! I'll be written in example.txt now")
}

You can also be absurd and close stdout. What happens then?

func main() {
    syscall.Close(1)
    fmt.Println("you'll never see me with stdout closed")
}

The program exits successfully, but nothing prints.

Now look what happens when we close stdout and then open a file

func main() {
    syscall.Close(1)

	file, err := os.Open("example.txt")
	if err != nil {
		panic(err)
	}
	defer file.Close()
	fmt.Fprintln(os.Stderr, "example.txt FD: ", file.Fd())
    // example.txt now has file descriptor 1
}

This is revealing. File descriptor 1 is not strictly reserved for stdout. The kernel uses the first available value.

Sockets have File Descriptors too.

You can think of a socket as “opening” a file on another computer, and your network interface card acting as an intermediary to facilitate data transmission.

func main() {
	ln, _ := net.Listen("tcp", "127.0.0.1:0")
	defer ln.Close()

	conn, err := net.Dial("tcp", ln.Addr().String())
	if err != nil {
		panic(err)
	}

	defer conn.Close()

	tcpConn := conn.(*net.TCPConn)
	socketFile, err := tcpConn.File()
	if err != nil {
		panic(err)
	}
	defer socketFile.Close()

	fmt.Printf("Socket File Descriptor: %d\n", socketFile.Fd())
}

Just like other processes, we can redirect the file descriptors of the socket. In fact this is the basis for how remote shells work. Run a shell on one machine, redirect input and output to read and write to the socket connected to your other machine.

func main() {
	ln, _ := net.Listen("tcp", ":8080")
	conn, _ := ln.Accept()

	cmd := exec.Command("/bin/sh")

	cmd.Stdin = conn
	cmd.Stdout = conn
	cmd.Stderr = conn

	cmd.Run()
}