UNIX_IPC

8/3/2019 UNIX_IPC

1/122

Interprocess

Communication in Unix

Department of Informatics Engineering

University of Coimbra

Copyright: Paulo Marques, DEI

An Introduction toConcurrent Programming

8/3/2019 UNIX_IPC

2/122

2

Contents

Concurrent Programming Concepts Process Creation and Management Signals Pipes, Named Pipes and Unix Sockets IO Multiplexing Message Queues Shared Memory Semaphores POSIX Threads

Creation and Management Synchronization

8/3/2019 UNIX_IPC

3/122

3

Standards

There are many UNIX flavors around... Linux, HP-UX, Solaris, AIX, BSD, IRIX, etc. Different capabilities, different APIs for advanced functionalities

Vendors tried to find a common ground: POSIX = Portable Operating System Interface

Standards? UNIX System V (Release 4) BSD 4.3 (1988) POSIX.1 (1994) POSIX Spec. 1170 (2001) IEEE Std. 1003.1-2001 (or POSIX)

8/3/2019 UNIX_IPC

4/122

4

POSIX Extensions

There is the 2001 base standard and theres its extensions Implementations that comply with the base standard define

_POSIX_VERSION to 200112L in

8/3/2019 UNIX_IPC

5/122

5

POSIX Extensions (2)

And many,many more...

8/3/2019 UNIX_IPC

6/122

6

Man pages typically indicate compliance

8/3/2019 UNIX_IPC

7/122

InterprocessCommunication in Unix




Process Creation andManagement

8/3/2019 UNIX_IPC

8/122

8

Process Model

Process creation in Unix isbased on spawning childprocesses which inherit all thecharacteristics of their fathers

Variables, program counter,open files, etc.

Spawning a process is doneusing the fork() system call

After forking, each process willbe executing having differentvariables and different state.

The Program Counter will bepointing to the next instruction

Changing a variable in the childprogram doesnt affect itsfather (and vice-versa)

a = f();fork();b = g();

State

a = f();

fork();b = g();

State

a = f();

fork();b = g();

State

ChildProcess

Original Process

OriginalProcess

8/3/2019 UNIX_IPC

9/122

9

Process Management

Each process has an unique identifier (PID). Each process has afather, which is also identified (PPID).

pid_t getpid(void);Returns the PID of the current process.

pid_t getppid(void);Returns the PID of the parent process.

pid_t fork(void);Creates a new process which inherits all its fathers state. It returns 0on the original process and the childs PID in the spawned process.

pid_t wait(int* status);Waits until a child process exits. The status of the child is set instatus. (status is the return value of the process)

pid_t waitpid(pid_t who, int* status, int options);Same as wait() but allows to wait for a particular child. In options, byusing WNOHANG in options, allows for checking if a child has alreadyexited without blocking. 0 in whomeans wait for any child.

8/3/2019 UNIX_IPC

10/122

10

Using processes for doing different things

The key idea is to create asymmetry between processes#include#include#include

#include

int main(){pid_t id;

id = fork();if (id == 0){

printf("[%d] I'm the son!\n", getpid());

printf("[%d] My parent is: %d\n", getpid(), getppid());}

else{

printf("[%d] I'm the father!\n", getpid());wait(NULL);

}return 0;

}

8/3/2019 UNIX_IPC

11/122

11

And the result is...

But why did we do wait(NULL)?

...printf("[%d] I'm the father!\n", getpid());

wait(NULL);...

8/3/2019 UNIX_IPC

12/122

12

Process Termination in UNIX

A process is only truly eliminated by the operating systemwhen its father calls wait()/waitpid() on it. This allows the parent check things like the exit code of its sons

Zombie Process: One that has died and its parent hasnot acknowledged its death (by calling wait()) Be careful with this if your are designing servers. They are eating

up resources!!

Orphan Process: One whose original parent has died. Inthat case, its parent becomes init (process 1).

8/3/2019 UNIX_IPC

13/122

13

Lets generate some Zombies

#include#include

#include#include#include

void worker() {printf("[%d] Hi, I'm a worker process! Going to die...\n",

getpid());}

int main(){

for (int i=0; i

8/3/2019 UNIX_IPC

14/122

14

Zombies (2)

8/3/2019 UNIX_IPC

15/122

15

Lets see adoption by init

#include (...)

void worker(){ sleep(10);printf("[%d] Let's see who my dady is: %d\n", getpid(),

getppid());}

int main(){for (int i=0; i

8/3/2019 UNIX_IPC

16/122

16


8/3/2019 UNIX_IPC

17/122

17

How to structure code

(...)

if ((id = fork()) == 0)

{// Huge amount of code// ...

}else

{ // Huge amount of code// ...

}

(...)

Fairly common...Dont do it!

8/3/2019 UNIX_IPC

18/122

18

How to structure code

void client(int id){// Client code

// ...}

if ((id = fork()) == 0){

client(id);exit(0);

}else if (id == -1){

error();}

// Original process code// ...

Server

Client Client Client

Note: You still have to consider how to take care of zombies

8/3/2019 UNIX_IPC

19/122

19

How a process becomes another executable

Somehow the OS must be able to execute code startingfrom an executable file e.g. how does the shell (bash) becomes ls?

int execl(const char *path, const char *arg, ...);int execlp(const char *file, const char *arg, ...);int execle(const char *path, const char *arg, ..., char *const envp[]);

int execv(const char *path, char *const argv[]);int execvp(const char *file, char *const argv[]);

exec family of functionsAllow to substitute the current process executable image by

another one. The substitution is complete!

The functions that have a p make use of the environment PATH;The functions that have a v make use of a pointer to an array onparameters; The functions that have an l have the parameterspassed separated by commas

Make sure that the first parameter is the name of the program!

8/3/2019 UNIX_IPC

20/122

20

Example

Simple program that lists the files in the current directory

Note: A successful exec() never returns The code, the stack, the heap, its all replaced by the new

executable

Original Code ls code

ls code

exec()

8/3/2019 UNIX_IPC

21/122

21

The corresponding code...

#include#include

#include#include

int main(){if (execlp("ls", "ls", "-a", NULL) == -1)

perror("Error executing ls: ");elseprintf("This cannot happen!\n");

return 0;

}

char* ls_param[] = { "ls", "-a", NULL };

if (execvp(ls_param[0], ls_param) == -1)

perror("Error executing ls: ");

Using an arraycan be moreflexible...

8/3/2019 UNIX_IPC

22/122





Asynchronous Events:Signals

8/3/2019 UNIX_IPC

23/122

23

Signals

A signal represents an asynchronous event which anapplication must (should? can?) process The programmer can register a routine to handle such events

Examples: The user hits Ctrl+C SIGINT The system requests the application to terminate SIGTERM The program tried to write to a closed channel SIGPIPE

Process

normal flowof execution

int sigint_handler() {// process signal}

SIGINT

8/3/2019 UNIX_IPC

24/122

24

Signals (2)

Signals can be in one of four states:

Blocked:Upon arrival, they are stored in a queue until the process unblocksthem. Then, they are delivered.

Ignored:Upon arrival, they are discarded. It is as if they had never existed.

Being Handled:They are redirected to a signal handler which is called.

None of the above:Non-handled, non-blocked or ignored signals. Upon arrival, theycause program termination.

Some signals cannot be ignored or handled (e.g. SIGKILL) When a process starts, signals are on their defaultbehavior.

Some are ignored, most are in the non-handled, non-blocked norignored state. If a signal occurs, the process will die.

8/3/2019 UNIX_IPC

25/122

25

Basic Signal Routines

typedef void (*sighandler_t)(int);

sighandler_t signal(int signum, sighandler_t handler);Redirects a certain signal (signum) to a handler routine.

int kill(pid_t pid, int sig);Sends a signal to a certain process identified by a PID. (Note: if pid is0, sends to all processes in the current process group.)

int pause();Blocks the process until a signal is received.

prototype of the

handler routine

int sigaction();int sigprocmask();int sigpending();int sigsuspend();

Note: These are the recommended

POSIX routines. We are not going tocover them here. The problem with

signal() is that in certain cases itsbehavior is undefined acrosssystems.

8/3/2019 UNIX_IPC

26/122

26

Handling a signal

void sigint(int signum) {char option[2];

signal(SIGINT, sigint);

printf("\n ^C pressed. Do you want to abort? ");scanf("%1s", option);if (option[0] == 'y') {printf("Ok, bye bye!\n");exit(0);

}}int main(){// Redirects SIGINT to sigint()

signal(SIGINT, sigint);

// Do some work!

while (1) {printf("Doing some work...\n");

sleep(1);}return 0;

}

8/3/2019 UNIX_IPC

27/122

27

Handling a signal (2)

8/3/2019 UNIX_IPC

28/122

28

Special constants in signal()

signal(SIGINT, SIG_IGN)

Ignores SIGINT

signal(SIGINT, SIG_DFL)

Restores SIGINT to itsdefault handling

8/3/2019 UNIX_IPC

29/122

29

The problem with signals

They make programming extremely hard Its completely asynchronous: you never know when you are going to get

a signal

This means that you have to protect all calls!

read(....);

After calling a standard function, it may return -1 indicating an error errno==EINTRmeans that a certain routine was interrupted and has to

be tried again.

Other routines return other things. It you are using signals, you have to protect them against all that!

8/3/2019 UNIX_IPC

30/122

30

The problem with signals (2)

For instance, simply to try to read a struct person fromdisk...

struct person p;...int n, total = 0;while (total < sizeof(p)){

n = read(fd, (char*)p + total, sizeof(p)-total);if (n == -1){if (errno == EAGAIN)

continue;else{

// True error!}

}total+= n;

}And you have to do something like thisfor all calls being done in your program!

8/3/2019 UNIX_IPC

31/122

31

Sending a signal

Its just a question of calling kill() with the PID of thetarget process...

void master(pid_t pid_son){printf("Master sleeping for a while...\n");sleep(3);printf("Master says: Hello son!\n");kill(pid_son, SIGUSR1);

}

int main() {pid_t son;

// Creates a worker processif ((son=fork()) == 0) {worker();exit(0);

}

// The mastermaster(son);wait(NULL);

return 0;}

8/3/2019 UNIX_IPC

32/122

32

The code of the child process...

void dady_call(int signum)

{printf("Dady has just called in!\n");

}

void worker(){// Redirect "user signal 1" to a handler routine

signal(SIGUSR1, dady_call);

// Do some workprintf("Child process, life is good...\n");for (int i=0; i

8/3/2019 UNIX_IPC

33/122

33


8/3/2019 UNIX_IPC

34/122

34

Danger!!!

What do you think it will happen if you receive a signal inside a signalhandler??

In most systems, upon entering a signal handling routine, all signals ofthat type become blocked (i.e. they are queued). [Well, for normalsignals, a finite set of them are queued (typically 1); for real timesignals, all are...]

The other signals are still processed asynchronously if they arrive. This behavior is not consistent across systems. In fact, in some systems,

that signal type resets to its default behavior. This means that if,meanwhile, the program receives a signal of the same type it may die! On

that type of system, the first thing that you must do is to once again setthe signal handler.

Well... doesnt really solve the problem, it just makes it less likely. The new POSIX routines address this use them. Also, most system

nowadays dont reset the signal handler.

void dady_call(int signum){signal(SIGUSR1, dady_call);

printf("Dady has just called in!\n");}

8/3/2019 UNIX_IPC

35/122

35

Beware

Signal numbers vary across operating systems andarchitectures. Dont rely on them, use symbolic constants

Linux, i386

man 7 signal

8/3/2019 UNIX_IPC

36/122

36

Some Signals

8/3/2019 UNIX_IPC

37/122





Pipes, Named Pipes andUNIX Sockets

8/3/2019 UNIX_IPC

38/122

38

Stream mode of communication

Pipes, Named Pipes and Stream Unix SOCKETS allowprocesses to communicate using streams of dataA pipe is a connection between two processes. You can send

things through the pipe, you can try to receive things from thepipe.

A pipe acts like a synchronous finite buffer. If a process tries to write to a pipe that is full, it blocks If a process tries to read from a pipe that is empty, it blocks

write() read()

pipe

8/3/2019 UNIX_IPC

39/122

39

Pipes

Provides for communication amount processes that arehierarchically related (i.e. father-child) Pipes must be created prior to creating child processes

Whenever a pipe is created, using pipe(), two filedescriptors are opened: one for reading (fd[0]), one forwriting (fd[1])

Unused file descriptors should be closed! Pipes are unidirectional (well... normally)

fd[1] fd[0]int fd[2];

pipe(fd);

readingwriting

8/3/2019 UNIX_IPC

40/122

40

Example

typedef struct {int a;int b;

} numbers;

// File descriptors for the pipe channelint channel[2];(...)

int main() {// Create a pipepipe(channel);

// Create the processesif (fork() == 0) {worker();exit(0);

}master();wait(NULL);

return 0;}

8/3/2019 UNIX_IPC

41/122

41

Example (cont.)

void worker() {numbers n;

close(channel[1]);

while (1) {read(channel[0], &n, sizeof(numbers));printf("[WORKER] Received (%d,%d) from master to add. Result=%d\n",

n.a, n.b, n.a+n.b);}

}

void master(){numbers n;

close(channel[0]);

while (1) {n.a = rand() % 100;

n.b = rand() % 100;

printf("[MASTER] Sending (%d,%d) for WORKER to add\n", n.a, n.b);write(channel[1], &n, sizeof(numbers));sleep(2);

}}

8/3/2019 UNIX_IPC

42/122

42

The result is...

8/3/2019 UNIX_IPC

43/122

43

Be careful!

A pipe is a finite buffer. If you try to write too much too quickly intoit, the process will block until some space clears up.

Atomicity is something to be dealt with If you try to write less that PIPE_BUF bytes into a pipe, you are

guarantied that it will be written atomically

It you try to write more, you have no guaranties! If several processes arewriting at the same time, the writes can be interleaved

Also, when a process tries to read from a pipe, you are not guarantiedthat it will be able to read everything Meaning...

You must synchronize your writes when youre writing a lot of data! You must ensure that you read complete messages!

struct person p;

int n, total = 0;

while (total < sizeof(p)) {n = read(fd[0], (char*)p + total, sizeof(p)-total);total+= n;

}

8/3/2019 UNIX_IPC

44/122

44

Controlling File Descriptors

Each process has a file descriptor table. By default, entries 0, 1 and 2are: stdin, stdout, stderr.

Each time a file is open, an entry is added to this table. Each time afile is closed, the corresponding entry becomes available.

The process descriptor table, in fact, contains only references to theOS global file descriptor table.

01

2

3

4

5

6

stdinstdout

stderr

f2

f3File Descriptor Table after:open(f1)open(f2)

open(f3)close(f1)

8/3/2019 UNIX_IPC

45/122

45

Controlling File Descriptors (2)

Two routines are useful for controlling file descriptors: int dup(int fd)Duplicates file descriptor fd on the first available position of the

file descriptor table.

int dup2(int fd, int newfd)Duplicates file descriptor fd on the newfd position, closing it ifnecessary.

Note that after a file descriptor is duplicated, the originaland the duplicate can be used interchangeably. Theyshare the file pointers, the buffers, locks, etc. Careful: Closing one file descriptor doesnt close all other that

have been duplicated!

8/3/2019 UNIX_IPC

46/122

46

Implementing a pipe between two processes

Implementing a pipe betweentwo processes is quite easy.Its only necessary to associatethe standard output of oneprocess with the standard inputof another.

Simple example: ls | sort. Note: closing one file

descriptor doesnt close allother that have beenduplicated!

8/3/2019 UNIX_IPC

47/122

47

Resulting in...

8/3/2019 UNIX_IPC

48/122

48

NAMED PIPES

8/3/2019 UNIX_IPC

49/122

49

Named Pipes (also known as FIFOs)

Similar to pipes but allow communication betweenunrelated processes. Each pipe has a name (string). The pipe is written persistently in the file system. For creating a named pipe, use the mkfifo command or call

mkfifo(const char* filename, mode_t mode);

Typically, like pipes, they are half-duplex Means that they must be open read-only or write-only They are opened like files, but they are not filesYou cannot fseek() a named pipe; write() always appends to the

pipe, read() always returns data from the beginning of the pipe.

After data is read from the named pipe, its no longer there. Itsnot a file, its an object in the unix kernel!

8/3/2019 UNIX_IPC

50/122

50

Unrelated client/server program (np_server.c)

#define PIPE_NAME "np_client_server"(...)

int main(){// Creates the named pipe if it doesn't exist yetif ((mkfifo(PIPE_NAME, O_CREAT|O_EXCL|0600)

8/3/2019 UNIX_IPC

51/122

51

Unrelated client/server program (np_client.c)

#define PIPE_NAME "np_client_server"(...)

int main(){// Opens the pipe for writingint fd;if ((fd = open(PIPE_NAME, O_WRONLY)) < 0) {perror("Cannot open pipe for writing: ");exit(0);

}

// Do some workwhile (1) {numbers n;n.a = rand() % 100;n.b = rand() % 100;printf("[CLIENT] Sending (%d,%d) for adding\n", n.a, n.b);write(fd, &n, sizeof(numbers));sleep(2);

}

return 0;}

8/3/2019 UNIX_IPC

52/122

52

Executing them...

8/3/2019 UNIX_IPC

53/122

53

Some interesting issues...

If you get a SIGPIPE signal, this means that you are tryingto read/write from a closed pipe

A named pipe is a connection between two processes. Aprocess blocks until the other party open the pipe...

Being it for reading or writing. Its possible to bypass this behavior (open it non-blocking

O_NONBLOCK), but be very, very careful: if not properlyprogrammed, it can lead to busy waiting. If a named pipe is opennon-blocking, EOF is indicated when read() returns 0.

When designing a client/server multiple client application, thismeans that either the pipe is re-opened after each client

disconnects, or the pipe is open read-write. If opened read-write, the server will not block until the other

party connects (since, he itself is also another party!)

8/3/2019 UNIX_IPC

54/122





I/O Multiplexing

8/3/2019 UNIX_IPC

55/122

55

Interesting Problem

A printer daemon is connected to a physical printer There are 3 named-pipes which allow automatic formatted

printing

PrinterDaemon

/printer/a4_double_sided

/printer/a4_single_sided

/printer/a3_single_sided

Pipes are blocking, so thisdoesnt work!!!

8/3/2019 UNIX_IPC

56/122

56

I/O Multiplexing

I/O Multiplexing: The ability to examine several filedescriptors at the same time select() and pselect()

int select(int n,fd_set* readfd,fd_set* writefd,fd_set* exceptfd,struct timeval* timeout)

Blocks until activity is detected or a timeout occurs.

Greatest fd plus one

For reading activity

For writing activity

For out-of-band activity

The fd_set variables are input/output. Upon return, they indicateif there was activity in a certain descriptor or not.

8/3/2019 UNIX_IPC

57/122

57

select()

Careful: n is the number of the highest file-descriptoradded of one. Its not the number of file descriptors

fd_setA bit set representingfile descriptors

FD_ZERO(fd_set* set)Cleans up the file descriptor set

FD_SET(int fd, fd_set* set)Sets a bit in the file descriptor set

FD_CLEAR(int fd, fd_set* set)Clears a bit in the file descriptor set

FD_ISSET(int fd, fd_set* set)Tests if a file descriptor is set

8/3/2019 UNIX_IPC

58/122

58

Example (printerd.c)

(...)

#define BUF_SIZE 4096#define NUM_PRINTERS 3

constchar* PRINTER_NAME[] = {"printer1", "printer2", "printer3"

};

// The printer file descriptorsint printer[NUM_PRINTERS];

void create_printers() {for (int i=0; i= 0);

}}

int main(int argc, char* argv[]) {create_printers();accept_requests();

}

8/3/2019 UNIX_IPC

59/122

59

Example (printerd.c) (2)

void accept_requests() {while (1) {fd_set read_set;

FD_ZERO(&read_set);for (int i=0; i 0 ) {for (int i=0; i 0) {buf[n] = '\0';printf("%s", buf);

}

} while (n > 0);

close(printer[i]);printer[i] = open(PRINTER_NAME[i], O_RDONLY|O_NONBLOCK);

}} } }

}

8/3/2019 UNIX_IPC

60/122

60

Resulting in...

8/3/2019 UNIX_IPC

61/122





Message Queues

8/3/2019 UNIX_IPC

62/122

62

Types of communication

Streams represent a flow of bytes. There are no fixeddata boundaries. The sender requests the transmission of N bytes The data starts flowing, the receiver starts getting it The receiver may get several chucks of less then N bytes

Messages represent a complete fixed structure of data Its like sending a letter. Either you get if fully or you dont. You

dont get half a letter.

169

8/3/2019 UNIX_IPC

63/122

63

Message Queues

Another IPC mechanism

Based on messages, not no data streams

Completely asynchronousA process can start executing, write some messages to a message

queue and die; another process can latter on come alive anreceive them.

Sharp contrast with all the mechanisms that weve seen so far,which require both the sender and the receiver to be present atthe same time

Message queues are maintained by the operating system. Theyare not destroyed if a process dies!

msgsnd() msgrcv()

8/3/2019 UNIX_IPC

64/122

64

Message Queues System V

int msgget(key_t key, int flags)Obtains an identifier to an existing message queue or creates a new

one. key can be IPC_PRIVATE (which creates a new unique identifier), or an

existing identifier. ftok() can be used to generate a number based on afilename.

flags, normal mode flags. When ORed with IPC_CREAT creates a newone.

int msgctl(int mqid, int cmd, struct msqid_ds* buff)Provides a variety of control operations on the message queue. mqid is the value returned by msgget() cmd is the command (most usually: IPC_RMID to remove it) buff a structure used in some control operations

8/3/2019 UNIX_IPC

65/122

65

Message Queues System V (2)

int msgsnd(int mqid, const void* message, size_t length, int flags)Sends a message to a certain key.

mqid is the value returned by msgget() message its a pointer to the message to send length represents the length of the payload of the message (not total) flags: 0 or IPC_NOWAIT (non-blocking)

int msgrcv(int mqid, void* message, size_t length, int type, int flags)Retrieves a message from a message queue. mqid is the value returned by msgget() message its a pointer to the message to receive length represents the maximum payload we are willing to receive type represent the type of message to receive (0 FIFO) flags: 0 or IPC_NOWAIT

8/3/2019 UNIX_IPC

66/122

66

Messages a Message Payload

In System V a message can be anything. But, it mustalways have a long integer in the beginning This long is called a message type identifier

typedef struct

{ long mtype;int first;int second;

} numbers_message;

Message type (must be >0)!

Payload

8/3/2019 UNIX_IPC

67/122

67

mq_pong.c (1)

typedefstruct {long mtype;

int first, second;} numbers_msg;

// Message queue idint id;

void cleanup(int signum) {

msgctl(id, IPC_RMID, NULL);exit(0);}

void main(int argc, char* argv[]) {assert( (id = msgget(IPC_PRIVATE, IPC_CREAT|0700)) != 0 );signal(SIGINT, cleanup);

if (fork() == 0)ping();

elsepong();

}

8/3/2019 UNIX_IPC

68/122

68

mq_pong.c (2)

void ping(){

numbers_msg msg;msg.first = rand() % 100;msg.second = rand() % 100;

while (1) {msg.mtype = 1;

printf("[A] Sending (%d,%d)\n", msg.first, msg.second);msgsnd(id, &msg, sizeof(msg)-sizeof(long), 0);

msgrcv(id, &msg, sizeof(msg)-sizeof(long), 2, 0);

printf("[A] Received (%d,%d)\n", msg.first, msg.second);

++msg.first;++msg.second;

sleep(3);}

}

8/3/2019 UNIX_IPC

69/122

69

mq_pong.c (3)

void pong(){

numbers_msg msg;

while (1) {msgrcv(id, &msg, sizeof(msg)-sizeof(long), 1, 0);printf("[B] Received (%d,%d)\n", msg.first, msg.second);

msg.mtype = 2;

++msg.first;++msg.second;

printf("[B] Sending (%d,%d)\n", msg.first, msg.second);

msgsnd(id, &msg, sizeof(msg)-sizeof(long), 0);}

}

R l i i

8/3/2019 UNIX_IPC

70/122

70

Resulting in...

IPC R

8/3/2019 UNIX_IPC

71/122

71

IPC Resources

Remember, IPC resources are not automatically cleaned This can lead so serious resource leaks

ipcs allows you to see the current System V IPCs in useipcrm allows you to manually delete resources

POSIX M Q

8/3/2019 UNIX_IPC

72/122

72

POSIX Message Queues

Very similar to System V except: Message types represent prioritiesA read from a POSIX message queue always returns the oldest

message of the largest type (priority)

POSIX message queues allow a signal to be raised when amessage is put on an empty queue or the initiation of a thread

Messages queues are represented by names on the file system(like named pipes)

mq_open() mq_close() mq_unlink() mq_send() mq_receive()

8/3/2019 UNIX_IPC

73/122





Shared Memory andSemaphores

Wh h d ?

8/3/2019 UNIX_IPC

74/122

74

Why shared memory?

Up until now... System calls are slow! Copying thought the kernel is slow!

P1 P2

buffer

write() read()

UserSpace

KernelSpace

Wh h d ?

8/3/2019 UNIX_IPC

75/122

75

Why shared memory?

Shared Memory (Almost) No kernel involvement! Fast! Very Fast!

P1 P2

shared

bufferUserSpace

Kernel

Space

Dangerous, very dangerous!

H d it k

8/3/2019 UNIX_IPC

76/122

76

How does it work

Each process has an address space Each address space corresponds to a page table. There are as

many page tables as there are processes

Shared memory corresponds to putting the same realmemory pages in the page tables of two differentprocesses

0 0

4Gb 4Gb

Address Space

Process A

Address Space

Process B

1000 1000

Physical Memory

0

256Mb5000

Page Translation

(slightly simplified)

Shared Memor S stem V

8/3/2019 UNIX_IPC

77/122

77

Shared Memory System V

int shmget(key_t key, int size, int flags)Obtains an identifier to an existing shared memory or creates a new

one. key can be IPC_PRIVATE (which creates a new unique identifier), or an

existing identifier. ftok() can be used to generate a number based on afilename.

size its the shared memory size in bytes flags, normal mode flags. When ORed with IPC_CREAT creates a new

one.

int shmctl(int shmid, int cmd, struct shmid_ds* buff)Provides a variety of control operations on the shared memory. shmid is the value returned by shmget() cmd is the command (most usually: IPC_RMID to remove it) buff a structure used in some control operations

Shared Memory System V (2)

8/3/2019 UNIX_IPC

78/122

78

Shared Memory System V (2)

int shmat(int shmid, const void* where, int flags)Maps a certain shared memory region into the current process

address space. shmid represents the shared memory identifier shmid returned by

shmget()

where represents an unused address space location where to map theshared memory (normally, use NULL)

flags represent different ways of doing the mapping (typically 0)

int shmdt(const void* where)Unmaps a certain shared memory region from the current addressspace. where represents an unused address space location where to map the

shared memory (normally, use NULL)

How does attaching work

8/3/2019 UNIX_IPC

79/122

79

How does attaching work

Address Space A Address Space B

Real Memory

int id = shmget(1234, 2, IPC_CREAT|0777)

char* p = shmat(id, NULL, 0);

p[0] = 19;

int id = shmget(1234, 2, 0777)

char* p = shmat(id, NULL, 0);

p[1] = 59;

p

p(id.. 1234)

19

19

19

59

5959

Whats wrong with this routine?

8/3/2019 UNIX_IPC

80/122

80

Whats wrong with this routine?

P1 P2

print_work(a, 12); print_work(b, 65);

Synchronization Semaphores

8/3/2019 UNIX_IPC

81/122

81


A semaphore is a synchronization object Controlled access to a counter (a value) Two operations are supported: wait() and post()

wait() If the semaphore is positive, decrement it and continue If not, block the calling process (thread)

post() Increment the semaphore value If there was any process (thread) blocked due to the semaphore, unblock one of

them.

5

P1

value

P6 P3 NULLblockedprocesslist

A semaphore

Corrected version

8/3/2019 UNIX_IPC

82/122

82

Corrected version

Mutual Exclusion:Only one process can be in here!

You always have to synchronize, even if you areonly reading or writing one byte!


8/3/2019 UNIX_IPC

83/122

83


System V Semaphores Works with semaphore arrays semget(), semctl(), semop()A little bit hard to use by themselves Use a library to encapsulate them!

POSIX Semaphores Quite easy to use sem_init(), sem_close(), sem_post(), sem_wait()Also work with threads! But... in Linux, they only work with threads (kernel 2.4)

Semaphores are very useful for other thingsbesides mutual exclusion: they can be used to count things!

semaphore h

8/3/2019 UNIX_IPC

84/122

84

semaphore.h

My Semaphore Library

Example Producer/Consumer

8/3/2019 UNIX_IPC

85/122

85


A producer puts elements on a finite buffer. If the bufferis full, it blocks until theres space.

The consumer retrieves elements. If the buffer is empty, itblocks until something comes along.

We will need three semaphores

One to count the empty slots One to count the full slots One to provide for mutual exclusion to the shared buffer

Producer Consumer


8/3/2019 UNIX_IPC

86/122

86


Producer Consumer

empty full

read_pos write_pos

put_element(e) {sem_wait(empty);sem_wait(mutex);

buf[write_pos] = e;write_pos = (write_pos+1) % N;sem_post(mutex);sem_post(full);

}

mutex

get_element() {sem_wait(full);sem_wait(mutex);

e = buf[read_pos];read_pos = (read_pos+1) % N;sem_post(mutex);sem_post(empty);return e;

}

The result of executing it!

8/3/2019 UNIX_IPC

87/122

87

The result of executing it!

And the main part of its code

8/3/2019 UNIX_IPC

88/122

88

And the main part of its code...

void producer() {for (int i=TOTAL_VALUES; i>0; i--) {printf("[PRODUCER] Writing %d\n", i);put_element(i);}}

void consumer() {for (int i=0; i

8/3/2019 UNIX_IPC

89/122

89

put_element() and get_element()

void put_element(int e) {sem_wait(sem, EMPTY);sem_wait(sem, MUTEX);buf[write_pos] = e;write_pos = (write_pos+1) % N;sem_post(sem, MUTEX);sem_post(sem, FULL);}

int get_element() {sem_wait(sem, FULL);sem_wait(sem, MUTEX);int e = buf[read_pos];read_pos = (read_pos+1) % N;sem_post(sem, MUTEX);sem_post(sem, EMPTY);return e;}

init() and terminate()

8/3/2019 UNIX_IPC

90/122

90

init() and terminate()

int sem, shmid;int write_pos, read_pos;

int* buf;

void init() {sem = sem_get(3, 0);sem_setvalue(sem, EMPTY, N); // N is the number of slotssem_setvalue(sem, FULL, 0);

sem_setvalue(sem, MUTEX, 1);

write_pos = read_pos = 0;

shmid = shmget(IPC_PRIVATE, N*sizeof(int), IPC_CREAT|0700);

buf = (int*) shmat(shmid, NULL, 0);}

void terminate() {

sem_close(sem);shmctl(shmid, IPC_RMID, NULL);

}

Remember

8/3/2019 UNIX_IPC

91/122

91

Remember

Always cleanup...

Implementation of semlib (semlib.c)

8/3/2019 UNIX_IPC

92/122

92

Implementation of semlib (semlib.c)

Implementation of semlib (semlib.c) (2)

8/3/2019 UNIX_IPC

93/122

93



8/3/2019 UNIX_IPC

94/122

94


8/3/2019 UNIX_IPC

95/122





POSIX Threads

Threads

8/3/2019 UNIX_IPC

96/122

96

Threads

Flows of execution inside of a program They share all the address space of a process Each thread has its own stack and local variables

(even so, they can access others threads variables)Process

Thread 1 Thread 2Thread 3

Why use threads?

8/3/2019 UNIX_IPC

97/122

97

y

They are very light weight compared to processes Light context switches Fast to create and terminate Fast to synchronize

Much easier to program than shared memory! Everything is already shared Be careful to synchronize accesses!

POSIX Threads Thread Management

8/3/2019 UNIX_IPC

98/122

98

g

Simple thread creation example (simple thread.c)

8/3/2019 UNIX_IPC

99/122

99

p p ( p _ )

Running the example

8/3/2019 UNIX_IPC

100/122

100

g p

gcc lpthread D_REENTRANT Wall fich.c o fich

Compiling with threads

8/3/2019 UNIX_IPC

101/122

101

p g

Linux

-D_REENTRANT is quite important in LinuxThreads (Kernel2.4)

It instructs the compiler to use special re-entrant routine functions If you dont... it ONLY appears to work, until you get in trouble!

gcc lpthread D_REENTRANT Wall fich.c o fich

Beware: Many routines are not re-entrant, they cannot be

directly used with threads since they use common storage

in an unsynchronized way (e.g. stktok())!

In some cases, there are re-entrant versions (e.g. strtok_r()).Check the manual! Dont trust common sense.

Example of a non-reentrant routine

8/3/2019 UNIX_IPC

102/122

102

p

What happens if this is called from two

different threads at the same time??

Things to beware of

8/3/2019 UNIX_IPC

103/122

103

g

Doesnt work, i is on the stack and constantly changing

Doesnt work, (1) after main() dies, its variables disappear

race condition with the starting threads; (2) main() dies everything dies!

If you need to terminate the main() thread...

8/3/2019 UNIX_IPC

104/122

104

y ()

This is OK!

Note: the other threads continue to execute.

Synchronization

8/3/2019 UNIX_IPC

105/122

105

Mutexes Provide mutual exclusion zones between threads In fact, these are just fast binary semaphores

POSIX Semaphores Used to signal events across threads Used to count objects in an synchronized way

Condition VariablesAllow a thread to block or to notify others on any condition Semaphores are a kind of condition variable:

the implicit condition is the semaphore being greater than 0

Mutexes

8/3/2019 UNIX_IPC

106/122

106

POSIX Semaphores

8/3/2019 UNIX_IPC

107/122

107

Producer/Consumer Revisited

8/3/2019 UNIX_IPC

108/122

108

Producer Consumer

Producer

Producer

prod_cons_threads.c

8/3/2019 UNIX_IPC

109/122

109

prod_cons_threads.c (2)

8/3/2019 UNIX_IPC

110/122

110

Which results in...

8/3/2019 UNIX_IPC

111/122

111

Synchronization Condition Variables

8/3/2019 UNIX_IPC

112/122

112

Condition variables allow the programmer to suspend ornotify a thread on any condition!

Synchronization Condition Variables (2)

8/3/2019 UNIX_IPC

113/122

113

Important rule:A condition variable always has an associated mutex.Always check the condition variable in mutual exclusion. The

mutex must be locked.

How does it work?Makes thread Acheck its condition

again

Synchronization Condition Variables (3)

8/3/2019 UNIX_IPC

114/122

114

The thread tests a condition inmutual exclusion. If the conditionis false, pthread_cond_wait()atomically releases de mutex

ANDwaits until someonesignals that the conditionshould be tested again.

When the condition issignaledANDthe mutex isavailable, pthread_cond_wait()atomically reacquires themutexANDreleases thethread.

pthread_cond_signal() indicatesthat exactly one blockedthread should test thecondition again. Note that thisis note a semaphore. If there isno thread blocked, the signal islost.

If all threads should re-check thecondition, usepthread_cond_broadcast(). Sincea mutex is involved, each onewill test it one at a time, inmutual exclusion.

Example

8/3/2019 UNIX_IPC

115/122

115

Suppose a buffer that can hold a maximum of N elements.When it is full, it should immediately be emptied. While

the buffer is being emptied, no thread can put things intoit.

Producer

CleanerProducer

Producer

bounded_buffer.c

8/3/2019 UNIX_IPC

116/122

116

bounded_buffer.c (2)

8/3/2019 UNIX_IPC

117/122

117

With 3 producers and one cleaner...

8/3/2019 UNIX_IPC

118/122

118

Condition Variables Rules (!)

8/3/2019 UNIX_IPC

119/122

119

The condition must always be tested with a while loop, never an if!Being unlocked out of a condition variable only means that thecondition must be re-checked, not that it is true

The condition must always be checked and signaled inside a lockedmutex.

While condition() may betrue while Thread B is executing,something may happen betweenthe time that the condition is signaledand Thread A is unblock (e.g. another

thread may change the condition)

Synchronization Some basic rules...

8/3/2019 UNIX_IPC

120/122

120

Never Interlock waits! Locks should always be taken in the same order Always release locks in the reverse order they have been

taken

sem_wait(A)sem_wait(B)

// Critical Section

sem_post(B)sem_post(A)

sem_wait(B)sem_wait(A)

// Critical Section

sem_post(A)sem_post(B)

Deadlock!

One way to assure that you always take locks in the sameorder is to create a lock hierarchy. I.e. associate a numberto each lock using a table and always lock in increasingorder using that table as reference (index).

Synchronization Some basic rules... (2)

8/3/2019 UNIX_IPC

121/122

121

Sometimes it is not possibleto know each order to takewhen locking (or usingsemaphores) Example: you are using two

resources owned by theoperating system. They arecontrolled by locks. Youcannot be sure anotherapplication is not using

exactly the same resourcesand locking in reverseorder.

In that case, usepthread_mutex_trylock() orsem_trywait() and back offif you are unsuccessful. Allow the system to make

progress and not deadlock

To Learn More

8/3/2019 UNIX_IPC

122/122

Advanced Programming in the UNIXEnvironment, 2nd Edition

by W. Richard Stevens, Stephen A. RagoAddison-Wesley, June 2005

UNIX Network Programming, Volume 2:Interprocess Communications, 2nd Ed. by W. Richard Stevens

Prentice Hall, August 1998

Unix Systems Programming:Communication, Concurrency and Threads,2nd Edition

UNIX_IPC

Documents

Transcript of UNIX_IPC