Interprocess Communication

Processes often need to share tasks or data. For example, one process may produce results that another consumes. At the same time, processes may not trust each other. In such cases, they isolate themselves but still require controlled communication channels.

A web browser (a process) needs to ask the networking stack (another process or part of the kernel) to send data. A compiler needs to send its output to a linker. A printer daemon needs to accept jobs from many different applications.

This tension between sharing and isolation is at the heart of IPC design.

Naive Implementation

The naive approach to deal with the problem of IPC is to have processes read and write to a file.

However, this is highly inefficient. Data already exists in memory, yet we redundantly write it to disk and read it back. Disk I/O is orders of magnitude slower than memory access.

This motivates more direct, memory-based communication methods.

Improvement: Shared Memory

The logical next step is Shared Memory. The OS carves out a region of RAM that multiple processes map into their own virtual address spaces. It's like building a shared park between the two cities, with gates directly from each city into it.

How it works: Process A calls shm_open() and mmap(). Process B opens the same named shared memory object and calls mmap(). The OS ensures they both point to the same physical pages of RAM.
The Good: This is the fastest IPC. Once the mapping is set up, there are no system calls involved in the actual data transfer. Process A just writes to a pointer, and Process B can immediately read from its own corresponding pointer. It's pure memory access speed.
The Problem (and why you called it "insecure"): It's a free-for-all. Without any control, Process A could be writing to the same location Process B is reading, leading to race conditions and corrupted data. It also creates a massive synchronisation problem.

Imagine two people trying to write on the same whiteboard at the same time. The result is a mess. Process A needs to know when it's safe to write, and Process B needs to know when new, valid data is ready. Shared memory doesn't provide this. It must be combined with another IPC tool (like a semaphore or a mutex) to coordinate access. This complexity makes it error-prone.

Improvement: Queues and Synchronization

Instead of raw shared memory, we can use in-memory queues. A queue is a First-In-First-Out (FIFO) data structure. A queue provides structure: one process writes, another reads.

A bounded buffer (your "limited size" queue) is the perfect solution. It naturally provides synchronization:

If the queue is empty and Process B tries to read, it must wait until data is written.
If the queue is full and Process A tries to write, it must wait until Process B reads some data out, freeing up space.

This waiting is the synchronization. It elegantly solves the coordination problem that plagues shared memory. No complex mutexes or semaphores needed (for the basic case).

This queue is, in essence, a Unix pipe.

Unix Pipes

A pipe is essentially a queue managed by the kernel. It is a small, fixed-size circular buffer located in the kernel's memory space. Because it's in the kernel, the OS can carefully control all access.

It has two file descriptors: one for input (read end) and one for output (write end).

This is the genius of the pipe. It reuses the familiar Unix "everything is a file" paradigm. The pipe() system call returns two file descriptors:

fd[0]: The read end.
fd[1]: The write end.

Processes can use standard, simple system calls: write(fd[1], data, len) to put data into the queue and read(fd[0], buffer, len) to take data out.

Here is how it deals with the synchronization problem:

A read() on an empty pipe blocks (waits) until at least one byte is written.
A write() to a full pipe blocks until space becomes available.
Getting EOF: The reader gets EOF when read() returns 0. This happens only when the writer closes its write end.
Breaking the pipe (EPIPE): If the reader closes its read end and the writer tries to write() again, the kernel sends the writer the SIGPIPE signal. The write() system call returns -1 with errno set to EPIPE. This prevents a process from writing data into the void.

Here is how this can be implemented manually:

Parent creates a pipe: int fd[2]; pipe(fd); The parent now owns two file descriptors, fd[0] (read) and fd[1] (write).
Parent calls fork(): This creates an identical child process. The child inherits copies of all the parent's open file descriptors. Now, both the parent and the child have their own references to fd[0] and fd[1].
The Critical Cleanup (Closing the right ends): To create a one-way data flow:
- If the parent will write, and the child will read:
  - Parent closes its read end: close(fd[0]); (It will only write to its fd[1]).
  - Child closes its write end: close(fd[1]); (It will only read from its fd[0]).
- If the parent will read, and the child will write, they close the opposite ends.

The cleanup is important for 2 reasons:

Resource conservation: File descriptors are a limited resource.
Correctness & EOF (End-Of-File): A read() call on a pipe will only return 0 (indicating EOF) when all references to the write end of the pipe have been closed. If the child forgets to close its copy of the write end, the parent's read() will sit there forever, waiting for data from a child that will never send more, because a write end still exists. The kernel can't signal EOF. This is a classic and subtle bug.

Kernel Control

Pipes and other IPC mechanisms run inside the kernel.

The kernel enforces Synchronization (blocking when necessary), Security (ensuring processes don’t interfere outside agreed channels) and Resource limits (queue size, descriptor counts).

Protocols and Formalization

Once communication is established, processes need protocols to structure their exchanges.

Protocols define:

Message formats.
Expected sequences of operations.

These can be modeled as state machines describing valid transitions between communication states. This translates into message transaction diagrams which are visual representations of how messages flow between processes.

Example: A simple "Get Temperature" protocol over a pipe.

Without a protocol: Process A writes "25". Process B reads "25". Is that a room temperature? A pressure reading? A command? The end of a larger number "2500"? Acknowledgment of a previous message?
With a simple protocol (defined by a state machine):
The protocol might define:
1. Message Format: All messages are 4 bytes. The first byte is a Type (e.g., 1=request, 2=response, 3=error). The next 3 bytes are the Data/Payload (e.g., for a temperature, it's a reading in 1/10ths of a degree).
2. Transaction Rule: A client sends a REQUEST message. The server must respond with either a RESPONSE or an ERROR message. The client must not send another request until it has received the response.

Message Transaction Diagram (a type of sequence diagram):

text

Client Process (Thermostat UI)        Server Process (Sensor Driver)
--------------------------------       -------------------------------
        |                                          |
        |---- REQUEST(Type=1, Data=0) ----------->| (Client asks for temp)
        |                                          |
        |                                          | (Server reads sensor)
        |                                          |
        |<--- RESPONSE(Type=2, Data=235) ---------| (Server replies: 23.5 deg)
        |                                          |
        | (Client updates display, can now send next request)
        |                                          |

Formal State Machine for the Client:
- State 1: IDLE. Can send a request. (Action: Send REQUEST → go to WAITING state).
- State 2: WAITING. Cannot send another request. (Event: Receive RESPONSE → go to IDLE state. Event: Receive ERROR → go to IDLE state. Event: Timeout → go to IDLE state and log error).

Broader IPC Mechanisms

Beyond pipes and shared memory, operating systems provide:

Message queues: kernel-managed queues with richer semantics.
Sockets: for communication across machines or within the same machine.
Signals: lightweight notifications between processes.
Semaphores/Mutexes: synchronization primitives to coordinate access.

Subscribe for monthly AI research/project updates.