Unix Systems Programming: Lab 7- Pipes and System V IPC:  messaging, semaphores, shared memory

Purpose and Rationale

The purpose of this lab is to allow students to become comfortable with the utilization of Unix pipes, named and unnamed, and introduce to System V IPC through semaphores, message queues and shared memory.

Resources

FAQ (submission instructions and other useful stuff)
Please look at lab5 FAQ for useful material on unnamed and named pipes.
If you are not in our course email list, please subscribe to the cspp51081 email list here:
        http://mailman.cs.uchicago.edu/mailman/listinfo/cspp51081

Lecture 7 is the primary source for information on System V IPC. You can find additional information in the manpages, online, as well as the following sources:

• Kerrisk,  The Linux Programming Interface, Chapters 45-48.
  
• Matthew and Stone, Beginning Linux Programming, Chapter 13
• Molay, Understanding Unix/Linux Programming, pp. 499-502 (Shared Memory) and 506-514 (Semaphores with Shared Memory)
• Stevens, Advanced Programming in the Unix Environment, section 14.7 (Message Queues), section 14.8 (Semaphores), and section 14.9 (Shared Memory)

All work should be done on a machine in the department's Linux cluster. You can refer to ssh for more information on how to log into a remote machine.

Marks Distribution

In this assignment you will have to complete 3 excercises. First, you will do two exercises (left previously from Lab 5): PART 1 - Unnamed pipes and PART 2 - Named pipes .

Part 1 - Unnamed Pipes	10 points
Part 2 - Named Pipes	10 points

Then for the third exercise you will choose ONE of three options:

LAB 7

Part 1 - Unnamed Pipes

• Step 1: background

  A pair of related processes can use an 'unnamed pipe' to pass information between them. When a pipe is created, the operating system sets up two file descriptors: one that can be read from and one that can be written to. Any data put into the pipe (by writing to the 'write' side of the file descriptor) by one process can be grabbed by another (by reading from the 'read' file descriptor). One common use of this function/structure is to create a pipe and write the stdout from one process to the pipe's write file descriptor and read from the pipe's read file descriptor to another program's stdin. We use this system all of the time:

  cat /etc/passwd | grep root
  

  We are 'piping' the stdout from 'cat /etc/passwd' to the stdin of 'grep root'. The program that usually sets up this pipe for you is the user shell. The program flow is as follows:

  program has two commands to run
  program sets up a pipe using 'pipe()'
  program forks
  the parent closes the pipe writer file descriptor
  the parent runs 'dup2()' to duplicate it's stdin to the pipe's reader
  the child closes the pipe reader file descriptor
  the child runs 'dup2()' to duplicate it's stdout to the pipe's writer
  both parent and child run 'execvp()' or similar to run the commands
  

• Step 2: my_upipe

  In this portion of the lab project, you will write a simple program that creates a pipe between two processes given on the command line. The syntax for this program is as follows:

  ./my_upipe "cat /etc/passwd" "grep root"
  

  You may want to borrow code from your gosh project to handle the tokenizing of the two commands given. We will test the following commands on CS linux boxes:

  gawaine:$ ./my_upipe "cat /etc/services" "more"
  # /etc/services:
  # $Id: services,v 1.4 1997/05/20 19:41:21 tobias Exp $
  #
  # Network services, Internet style
  ...
  ...
  --More--
  ...
  ...
  
  gawaine:$ ./my_upipe "cat /etc/passwd" "grep root"
  root:x:0:0:Super-User:/:/bin/bash
  
  gawaine:$ ./my_upipe "ls -a -l /usr/bin" "wc"
   501 4668 32853
  

• Step 3: help
  
  There are example programs that use 'pipe' in the BLP. Also, you can look in /home/mark/pub/51081/pipes for some example code.
  

Step 4: See Deliverables


Part 2 - Named Pipes

• Step 1: background

  A pair of unrelated processes can use a 'named pipe' to pass information between them. Unlike 'unnamed pipes', however, 'named pipes' are accessed as a file on the file system. This allows a situation where two processes started in separate shells can communicate with eachother through a 'named pipe' on the file system. A named pipe, or FIFO, can be created using the 'mkfifo()' function. It can be removed (like any other file on the file system) using the 'unlink()' function. Once a named pipe file exists, programs can open it like they would other files and then use the file descriptor obtained to perform regualar file IO operations on the (read, write, close...).

• Step 2: my_npipe

  You will write two simple programs 'my_npipe_reader.c' and 'my_npipe_writer.c' that use a named pipe to communicate. The 'my_npipe_reader' program will set up a named pipe using 'mkfifo()', open it read only, and read strings from it until it recieves the string 'exit'. The writer will open the named pipe file, read strings from the user and write them to the named pipe. When the user enters 'exit', the program will write the string to the pipe and then exit. Execution should look something like this (note that you must start the reader first):

  reader:
  gawaine:$ ./my_npipe_reader
  Creating named pipe: /tmp/mypipe
  Waiting for input...Got it: 'hello world'
  Waiting for input...Got it: 'foober goober'
  Waiting for input...Got it: 'exit'
  Exiting
  
  
  writer:
  gawaine:$ ./my_npipe_writer
  Opening named pipe: /tmp/mypipe
  Enter Input: hello world
  Writing buffer to pipe...done
  Enter Input: foober goober
  Writing buffer to pipe...done
  Enter Input: exit
  Writing buffer to pipe...done
  Exiting
  

  Note that the 'my_npipe_reader' and 'my_npipe_writer' need to be executed in separate shells at the same time. The reader stops at 'Waiting for input...' until it recieves data from the pipe (the read completes).

• Step 3: Help

  Use BLP as a reference, there are many good code examples for named pipe usage. Also, see /home/mark/pub/51081/pipes for example code.

• Step 4: See Deliverables
  

Part 3 - Choose ONE excercise:

• EXERCISE 1: Client-Server Communication using Message Queues

  
  

  Problem Statement:

  Two processes, client and server, communicate via two message queues "Up" and "Down".

                          Server
  
                          ^     |
                     Up |       v Down
  
                          Client
  

  The client reads a message from the standard input and sends it to the server via the Up queue, then waits for the server's answer on the Down queue. The server is specialized in converting characters from lower case to upper case and vice versa. Therefore, if the client sends the message "lower case" via the Up message queue, the server will read the message, convert it, and send "LOWER CASE" via the Down queue. When the client receives the message from the server, it prints it out.  You may assume the maximum size of any message is 256 bytes.

  Multiple clients must be able to connect to the up and down queues.  However, what happens if one client puts a letter to convert on the system and another client is waiting for it's response from the queue?  There are different ways to handle this, but you should handle this using typed messages.  Each client has a particular client number based on the last 4 digits of its process ID.  Use this 4-digit number as the client identifier, and use typed messages on the queue so that each client can always be sure to receive the letter that it is waiting on to be converted.
   

  What to do:

  Implement the client and server from the scenario above.

  • Make sure you understand the problem. Send questions to the list. An example of how you could run two processes is below:

  hangao@gawaine:LAB7% server &
  hangao@gawaine:LAB7% client
  Insert message to send to server: message
  Msg processed: MESSAGE
  
  Insert message to send to server: UPPER CASE
  Msg processed: upper case
  

  • Write two C programs client.c and server.c that implement the behavior explained above. You may want to use this conversion function for the server code:

  /* convert upper case to lower case or vise versa */
  void conv(char *msg, int size)
  {
  

       for (i=0; i<size; ++i)
          if (islower(msg[i]))
              msg[i] =  toupper(msg[i]);
          else if (isupper(msg[i]))
              msg[i] =  tolower(msg[i]);
  }
   

  Deliverables

  This will complete step 4 of Deliverables:

  1. Create a directory ex3.1
  2. This directory will contain three files:
     • client.c
     • server.c
     • Makefile: which must will build your client and server. It must also contain a target clean.

  Grading

  This assignment is worth 20 points. Your Server must be able to handle multiple clients correctly to recieve full credit.

• EXERCISE 2: The Consumer-Producer Problem using Shared Memory

  
  

  Problem Statement

  Write 2 programs, producer.c implementing a producer and consumer.c implementing a consumer, that do the following:

  • Your product will sit on a shelf: this will be a shared memory segment that contains an integer, a count of the items "on the shelf". This integer may never drop below 0 or rise above 5.
  • Your producer creates the shared memory segment and sets the value of the count to 5. It is the producer program's responsibility to stock product on the shelf, but not overstocked. The producer may add one item to the shelf at a time, and must report to STDOUT every time another item is added as well as the current shelf count.
  • Your consumer will remove one item from the shelf at a time, provided the item count has not dropped below zero. The consumer will decrement the counter and report the new value to STDOUT. Have your consumer report each trip to the shelf, in which there are no items. For example, "Blasted!! I knew I should have gone to T---t."
  • Your program must work with multiple consumers: The item count should not fall below 0 or rise above 5 when there are several consumers and one producer. It is okay (and in fact expected) that your producer will have a hard time keeping-up with the demand of your consumers. Your program is not expected to keep-up with consumer demand.

  What to do

  Look on these examples of shared memory usage: shm_server.c creates a shared memory segment and writes "hello world" into it. shm_client.c reads and prints out the content of the shared memory segment. The way to compile and run the examples is:
  

  hangao@gawaine:LAB7% gcc -o shm_server shm_server.c
  hangao@gawaine:LAB7% gcc -o shm_client shm_client.c
  
  hangao@gawaine:LAB7% shm_server 1234 &
  [1] 27633
  Try to create this segment
  shared memory content: hello world
  
  hangao@gawaine:LAB7% shm_client 1234
  Trying shared memory 1234
  shared memory: 1152
  shared memory: 0x40016000
  shared memory content: hello world
   

  Deliverables

  This will complete step 4 of Deliverables:

  1. Create a directory ex3.2
  2. This directory will contain three files:
     • producer.c
     • consumer.c
     • Makefile: which must will build your client and server. It must also contain a target clean.

  Grading

  This assignment is worth 20 points. Your Producer will be run with multiple consumers. Your product count must not drop below 0 or rise above 5. Your program is not responsible for keeping the consumers happy: as more consumers are added, there will be more trips down an empty aisle.

• EXERCISE 3: Synchronization of Consumers and Producers using Semaphores

  
  

  Problem Statement

  You should read Exercise 2 carefully first. I recommend you implement exercise 2 before beginning the Bonus Problem.

  We will make three modifications to Exercise 2:

  • Consumers are now greedy, they will grab items off the shelf even if there are no items on the shelf (they grab a competitive brand.) Now, you shelf count can go drop below zero, but it may not rise above five. As more consumers are added, your poor producer cannot keep-up, so you will use semaphores to control your consumers.
  • To help your poor swamped single producer, allow for multiple producers. You may want to increase your product count, and experiment with serveral producers and consumers.
  • Producers are overly diligent and will try to overstock. They will always try to place an item on the shelf. Use semaphores to control your producer's behavior. Try to maintain your stock count regardless of how many producers and consumers: you may need to adjust the stock count to make it interesting.
  • THIS IS NOT PART OF THE ASSIGNMENT: Try having multiple consumers and producers, where each sleeps for a random period, and tries the aisle. How do your semaphores hold up?

  What to do

  Start by making sure you have a program which works for exercise 2. Now, up the ante, slowly and see if you can maintain sanity. Try each addition above to your program and see how it behaves. If your program has difficulty try to fix the problem, if you cannot analyze where you think the problem lies. Credit for this problem will not be solely based on successful performance, but on careful observation of your program's performance and analysis of difficulties you may be having controling behavior with semaphores. Keep a log of what you have and tried, and how your program has performed. This will be submitted as a README file.

  Deliverables

  This will complete step 4 of Deliverables:

  1. A directory ex3.2 with your files from exercise 2
  2. Create a directory bonus
  3. This directory will contain four files:
     • producer.c
     • consumer.c
     • Makefile: which must will build your client and server. It must also contain a target clean.
     • README: This should state how you have implemented your semaphare, whether your consumers can be greedy, your producers overly diligent, and how your program has fared under varying conditions. Careful notes will help your score here.

  Grading

  The bonus is worth 5 points total. Partial credit will be given. The purpose of this bonus is for you to kick back and see how System V IPC fairs under the strenuous conditions imposed by the Producer/Consumer Problem. Incremental points will be given, and success judged as much by your observations as by your program's performance. Your README file will be important here.