Remote Linux Tutorial

Students come into CS 152 with different levels of experience with Linux. The purpose of this lab is to make sure everyone has the basic Linux skills needed to succeed in this course. Everyone should run the commands in the Setting Up Your CMSC 15200 Directory section and review the Git Refresher section. Other than that, feel free to skip the sections that cover material that you already know well.

Objectives

  1. Learn basic terminal commands and how to work with a text editor

  2. Become familiar with the Linux environment

  3. Learn to compile and run a C program from the command-line

  4. Learn about file permissions

  5. Learn about redirection and pipes

Linux

Linux is an operating system much like macOS or Windows. It has windows, programs, web browsers, and so on. Files are stored in directories (folders) that, in turn, are stored in other directories. Although you can access Linux’s features using your mouse, as you perform more and more complex tasks, you will find that using the mouse is ineffective. Linux allows us to interact with the computer entirely through text using a program called the terminal. (Mac provides a similar terminal application, and there are ways to use text-based commands on Windows too. But, Linux provides the lowest barrier to entry.) In this lab you will learn how to use the terminal remotely to perform some basic operations in Linux. You will need these skills for the rest of your time at UChicago.

We show many examples of sample output below. The output you see when you run the commands may vary a bit.

Terminal/Shell

On your personal computer, you probably navigate your hard drive by double clicking on icons. While convenient for simple tasks, this approach is limited. For example, imagine that you want to delete all of the music files over 5 MB that you haven’t listened to in over a year. This task is very hard to do with the standard double-click interface but is relatively simple using the terminal.

The program that runs within a terminal window and processes the commands the you type is called a shell. We use bash, which is the default shell on most Linux distributions, but there are other popular shells, such as ksh, tcsh, etc.

Connecting to the CS Linux Servers

To get started, please use Visual Studio Code to connect to the CS Linux servers to do this lab. If you have not already installed Visual Studio Code (hereafter referred to as VSCode) and set up ssh, please follow the instructions on the Working Remotely with Visual Studio Code and SSH page. Once you have completed that task, follow the instructions for Using the terminal.

When you are connected, your VSCode window should look similar to this:

../_images/vscode_term1.png

The string amr@linux2:~$ is known as the prompt. Your prompt will look a bit different, for example, it will have your username rather than amr and you may be logged into one of the other Linux servers instead of linux2. Also, you may see a welcome message in the top pane.

When you start typing in the terminal window, the characters you type will appear to the right of the $.

Editing, Compiling, and Running C Programs

In this section, you will have a chance to edit, compile, and run a C program. Let’s start by looking at the code. Open the file hello.c in the editor pane of VSCode by running:

$ code hello.c

in the Linux terminal window.

If the file is empty, then you are likely in the wrong directory; Use cd to navigate to ~/cmsc15200-spr-21/username/lab0.

This classic program prints “hello world” to the screen. To run the program, you first need to compile it. We’ll be using a popular C compiler named clang. The most basic way to compile a C program using clang is to run the command:

$ clang hello.c
$

As you can see, this command finishes silently when there are no syntax errors in the program. The compiler will create an executable program named a.out by default. Here’s how to run this program at the command-line:

$ ./a.out
hello world
$

We will typically run the compiler with many more options:

$ clang -Wall -g -O2 hello.c -o hello
$

The arguments that start with a dash (-) are called options or flags.

  • The -Wall flag enables all compiler warnings. Even though the warnings will be irritating at times, do not ignore them! The warnings identify code that while technically legal in C may be incorrect or behave in unexpected ways.

  • The -g flag indicates that the compiler should include information for the debugger.

  • The -O0 flag tells the compiler not to optimize your code (which makes it easier to debug with the debugger). For production code, you would use a higher-level of optimization (O2, O3, etc).

  • The -o hello option tells the compiler to name the executable hello, rather than a.out, which is the default.

To run the new executable, run:

$ ./hello
hello world

For most assignments, we will provide a Makefile that will handle the work of calling the compiler with the correct options for you. The make utility will only create the executable if it does not exist or of the source file (hello.c in the case) has changed since the executable was last created. Since we just generate the executable and have not changed the file, we’ll see the following when we run make:

$ make hello
make: 'hello' is up to date.

If we remove hello and then re-run make, it will call the compiler for us:

$ rm hello
$ make hello
clang -g -Wall -O0 hello.c -o hello

C compilers often generate intermediate files that clutter up your directory. Our Makefile will usually include a target (we’ll explain the language of make later in the quarter) named clean for cleaning up these files and along with the executable. Just run:

$ make clean
rm -f *.o
rm -rf *dSYM
rm -f hello my_echo

to take advantage of this target.

Exercises

Try the following tasks to practice and check your understanding of these terminal commands.

  1. Compile hello.c by hand using clang and then run the resulting executable. Did you give it a name with the (-o flag)?

  2. Run make clean to remove the executable you just created.

  3. Run make hello and then run hello. Did it generate the output you expected?

  4. Edit hello.c to replace “hello world” with “hello <your name>”. Save the file and then compile and run it. Did you get it right or did you introduce a syntax error? If you got an error see if you can figure out what might be wrong and try again. (Ask for help if you get stuck.) If you got it right the first time, try leaving out one of the double quotes (") or the semicolon at the end of line 4 and see what happens.

  5. Run make clean to cleanup when you are finished.

Git Refresher

Most students coming into CS 152 will have used the Git version control system and our GitLab server in a previous course. If you have not used Git, please let us know and we will make an appointment with you to explain the basics of Git.

For those of you who have used Git, here is a quick refresher.

  • Run git status . to get the status of the files in the current directory from git’s perspective.

  • Run git add <filename> to add a new file or file that has been updated to the list of files to be saved on the next commit. (<filename> should be replaced by the names of one or more files.)

  • Run git commit -m"Some message" to create a commit from the files that you added since the last commit.

  • Run git push to send your commit to the server and

  • Run git pull to pull the most recent copy of the files from the server to your local machine.

Finally, when you ran the git commands earlier, we connected your personal repository to an upstream repository that we will use to distribute files. You will run the command git pull upstream master to pull files from the upstream repository into your own.

In this course, we will stick to the basics. Your workflow will be as follows:

  • Log into a CS machine using VSCode and ssh.

  • Change to your cmsc15200-spr-21/username directory

  • Pull updates from the Git server (we will add files to your repository throughout the quarter).

  • Do your work

  • Add files as needed, using git status . to help you remember what you’ve changed.

  • Create a commit with any changes you have made

  • Push the commit to the Git server.

Remember that the course staff does not have access to any files stored in your home directory or files on your laptop. All we can access are files that have been pushed to the Git server, so remember to always push your latest commits when you’re done working for the day or when you ask a question on Ed that will require us to look at your code.

If you are looking to learn more about Git, we recommend the Version Control lecture from the MIT Missing Semester Course as a good starting point.

Wild Cards (using an asterisk)

Sometimes when we enter a string, we want part of it to be variable, or a wildcard. A common task is to list all files that end with a given extension, such as .txt. The wildcard functionality, through an asterisk, allows you to simply say:

$ ls *.txt

The wildcard can represent a string of any length consisting of any characters - including the empty string.

It is important to be careful using wildcard, especially for commands like rm which cannot be undone. A command like:

$ rm *             ### DO NOT RUN THIS COMMAND!

will delete all of the files in your working directory!

Exercises

  1. What do you expect to see when you run the command ls *.c from within your cmsc15200-spr-21/username/lab0 directory?

  2. What do you expect to see when you run the command ls *e* from within your cmsc15200-spr-21/username/lab0 directory?

Useful Keyboard Shortcuts

Used at the Linux prompt, the keyboard shortcut Ctrl-P will roll back to the previous command. If you type Ctrl-P twice, you will roll back by two commands. If you type Ctrl-P too many times, you can use Ctrl-N to move forward. You can also use the arrow keys: up for previous (backward), down for next (forward).

Here are a few more useful shortcuts:

  • Ctrl-A will move you to the beginning of a line.

  • Ctrl-E will move you to the end of a line.

  • Ctrl-U will erase everything from where you are in a line back to the beginning.

  • Ctrl-K will erase everything from where you are to the end of the line.

  • Ctrl-L will clear the text from the current terminal

Play around with these commands. Being able to scroll back to, edit, and then rerun previously used commands saves time and typing! And like auto-completion, getting in the habit of using keyboard shortcuts will reduce frustration as well as save time.

Man Pages

A man page (short for manual page) documents or describes topics applicable to Linux programming. These topics include Linux programs, certain programming functions, standards, and conventions, and abstract concepts.

To get the man page for a Linux command, you can type:

man <command name>

So in order to get the man page for ls, you would type:

man ls

This command displays a man page that gives information on the ls command, including a description, flags, instructions on use, and other information.

Each man page has a description. The -k flag for man allows you to search these descriptions using a keyword. For example:

man -k printf

This command searches all the descriptions for the keyword printf and prints the names of the man pages with matches.

Type:

q

to exit from a man page.

Combining Commands

Running Commands Sequentially

It is often convenient to chain together commands that you want to run in sequence. For example, recall that to print the working directory and list all of the files and directories contained inside, you would use the following commands:

$ pwd
/home/username/
$ ls
Desktop  Documents  Downloads  Music  Pictures  Public  Templates  Videos

You could also run them together, like so:

$ pwd ; ls
/home/username/
Desktop  Documents  Downloads  Music  Pictures  Public  Templates  Videos

First, pwd is executed and run to completion, and then ls is executed and run to completion. The two examples above are thus equivalent, but the ability to run multiple commands together is a small convenience that could save you some time if there is a group of commands that you want to execute sequentially.

Note

The shell doesn’t care about white space, so it will run any of the following as well:

$ pwd;ls
$ pwd ;ls
$ pwd; ls
$ pwd       ;        ls

Redirection

The examples in this section will use commands that we’ve not yet discussed. Refer to the man pages for information about unfamiliar commands.

As we already know, commands like pwd, ls, and cat will print output to screen by default. Sometimes, however, we may prefer to write the output of these commands to a file. In Linux, we can redirect the output of a program to a file of our choosing. This operation is done with the > operator.

Try the following example from within your lab0 directory and compare your output with ours:

$ pwd
/home/username/cmsc15200-spr-21/username/lab0
$ touch test-0.txt
$ ls > test-1.txt
$ cat test-1.txt
backups
hello.c
test-0.txt
test-1.txt
test.txt
$ echo "Hello World!" > test-2.txt
$ cat test-2.txt
Hello World!
$ cat test-2.txt > test-1.txt; cat test-1.txt
Hello World!
$ rm test-*

Two important things to note:

  1. If you redirect to a file that does not exist, that file will be created.

  2. If you redirect to a file that already exists, the contents of that file will be overwritten.

You can use the append operator (>>) to append the output of a command to the end of an existing file rather than overwrite the contents of that file.

Not only can we redirect the output of a program to a file, we can also have a program receive its input from a file. This operation is done with the < operator. For example:

$ make my_echo
clang -g -Wall -O0 my_echo.c -o my_echo
$ my_echo < my-input.txt

In general, all Linux processes can perform input/output operations through, at least, the keyboard and the screen. More specifically, there are three ‘input/output streams’: standard input (or stdin), standard output (or stdout), and standard error (or stderr). The code in my_echo.c simply reads information from stdin and writes it back out to stdout. The redirection operators change the bindings of these streams from the keyboard and/or screen to files. We’ll discuss stderr later in the term.

Piping

In addition to the ability to direct output to and receive input from files, Linux provides a very powerful capability called piping. Piping allows one program to receive as input the output of another program, like so:

$ program1 | program2

In this example, the output of program1 is used as the input of program2. Or to put it more technically, the stdout of program1 is connected to the stdin of program2.

As another more concrete example, consider the man command with the -k option that we’ve previously discussed. Let’s assume that you hadn’t yet been introduced to the mkdir command. How would you look for the command to create a directory? First attempts:

$ man -k "create directory"
create directory: nothing appropriate
$ man -k "directory"
(a bunch of mostly irrelevant output)

As we can see, neither of these options is particularly helpful. However, with piping, we can combine man -k with a powerful command line utility called grep (see man pages) to find what we need:

$ man -k "directory" | grep "create"
mkdir (2)            - create a directory
mkdirat (2)          - create a directory
mkdtemp (3)          - create a unique temporary directory
mkfontdir (1)        - create an index of X font files in a directory
mklost+found (8)     - create a lost+found directory on a mounted Linux second extended fil...
mktemp (1)           - create a temporary file or directory
pam_mkhomedir (8)    - PAM module to create users home directory
update-info-dir (8)  - update or create index file from all installed info files in directory
vgmknodes (8)        - recreate volume group directory and logical volume special files

Nice.

Exercises

  1. Use piping to chain together the printenv and tail commands to display the last 10 lines of output from printenv.

  2. Replicate the above functionality without using the | operator. (hint: Use a temporary file.)

File Permissions

Sometimes we want to restrict who can access certain resources on the file system.

Most file systems assign ‘File Permissions’ (or just permissions) to specific users and groups of users. Unix is no different. File permissions dictate who can read (view), write (create/edit), and execute (run) files on a file system.

All directories and files are owned by a user. Each user can be a member of one or more groups. To see your groups, enter the command groups into the command line.

File permissions in Unix systems are managed in three distinct scopes. Each scope has a distinct set of permissions.

User - The owner of a file or directory makes up the user scope.

Group - Each file and directory has a group assigned to it. The members of this group make up the group scope.

Others - Every user who does not fall into the previous two scopes makes up the others scope.

If a user falls into more than one of these scopes, their effective permissions are determined based on the first scope the user falls within in the order of user, group, and others.

Each scope has three specific permissions for each file or directory:

read - The read permission allows a user to view a file’s contents. When set for a directory, this permission allows a user to view the names of files in the directory, but no further information about the files in the directory. r is shorthand for read permissions.

write - The write permission allows a user to modify the contents of a file. When set for a directory, this permission allows a user to create, delete, or rename files. w is shorthand for write permissions.

execute - The execute permission allows a user to execute a file (or program) using the operating system. When set for a directory, this permission allows a user to access file contents and other information about files within the directory (given that the user has the proper permissions to access the file). The execute permission does not allow the user to list the files inside the directory unless the read permission is also set. x is shorthand for execute permissions.

To list information about a file, including its permissions, type:

ls -l <filepath>

You’ll get output of the form:

<permissions> 1 owner group <size in bytes> <date modified> <filepath>

For example, if we want information on /usr/bin/python3.5:

$ ls -l /usr/bin/python3.5
-rwxr-xr-x 1 root root 4460272 Aug 20 /usr/bin/python3.5

First thing we can notice is that the owner of the file is a user named root. (FYI, root is a name for an account that has access to all commands and files on a Linux system. Other accounts may also have “root” privileges.) The file’s group is also root.

The permissions are -rwxr-xr-x. The initial dash (-) indicates that /usr/bin/python3.5 is a file, not a directory. Directories have a d instead of a dash. Then the permissions are listed in user, group, and others order. In this example, the owner, root, can read (r), write (w), and execute (x) the file. Users in the root group and all other users can read and execute the file.

Exercises

By default, any files or directories that you create will have your username as both the user and the group. (If you run groups, you’ll notice that there is a group with the same name as your username. You are the only member of this group.) On our Linux machines, by default, new files are given read and write permissions to user and group and no permissions to other. New directories will be set to have read, write and execute permissions for user and group.

  1. Verify this claim by running ls -l backups/copy2.txt and ls -ld  backups in your lab0 directory.

The -d flag tells ls to list the directory, instead of its contents. Notice that the first letter in the permissions string for backups is a d, while it is a - for backups/copy2.txt.

Once you have verified the claim, go ahead and remove the backups directory using the command: rm -r backups.

Changing Permissions, Owner, & Group

chmod <permissions> <path-name>

set the permissions for a file/directory

chmod <changes> <path-name>

update the permissions for a file/directory

chown <username> <path-name>

change the owner of a file to username

chgrp <group> <path-name>

change the group of a file

cat <path-name>

print the contents of a file to the terminal

To change permissions, we use the chmod command. There are two ways to specify the permissions. We’ll describe the more accessible one first: to set the permissions you specify the scope using a combination of u, g, and o, the permission using r, w, and x, and either + or - to indicate that you want to add or remove a permission. For example uo+rw indicates that you want to add read and write permissions for the user and others groups.

We can demonstrate this using the cat command to print file contents to the terminal:

$ echo "Hello!" > testfile
$ ls -l testfile
-rw-rw---- 1 username username 7 Aug 23 11:22 testfile
$ cat testfile
Hello!
$ chmod ug-r testfile #remove read permissions from user and group
$ ls -l testfile
--w--w---- 1 username username 7 Aug 23 11:22 testfile
$ cat testfile
cat: testfile: Permission denied
$ chmod u+r testfile #give user scope read permissions

In this last example, we have added user read permissions to testfile.

In addition to the symbolic method for setting permissions, you can also use a numeric method: each permission has a unique value: read = 4, write = 2, execute = 1. As a result, you can describe the permissions of each scope using the sum of its permissions’ values. For example, if a file has read and write permissions for the user scope, its permissions can be described as 6 (4 + 2 = 6).

You can describe the permissions of a file overall using these values for each scope. For example, 761 describes the permissions for a file with read, write, and execute permissions for the user scope, read and write permissions for the group scope, and only execute permissions for the others scope.

The symbolic approach is relative: it allows you to add and remove permissions relative to the current file permissions. The numeric method is absolute: it sets the permissions to a specific configuration. We recommend starting with the symbolic approach. It is easier to get right. As you get more comfortable with setting permissions, it is useful to learn how to use the numeric method.

To change the owner of a file or directory (if you are the owner or root), use the command:

chown <new owner> <path to file>

To change a file’s group (if you are the owner or root), use the command:

chgrp <new group> <path to file>

It is unlikely that you will need to use these two commands for this course.

Exercises

  1. Run echo "Hello!" > testfile to construct testfile. Look at the permissions using ls -l.

  2. Change the permissions on testfile to allow read access for others. Run ls -l testfile to check the new permissions.

  3. Remove group write access from testfile. Check the corrected permissions.

  4. Remove testfile using rm.

Final Notes

Sometimes, a program will run indefinitely or misbehave. When this happens, you can type Ctrl-C to send an interrupt signal to the running program, which usually causes it to terminate. On occasion, you may need to type Ctrl-C a few times. Typing Ctrl-D sends an end of input signal, which tells the program that no more information is coming.

Clean up

It is always a good idea to clean up your workspace before you exit VSCode and close your ssh connection.

  1. Run make clean to remove any intermediate files and executables that you created.

  2. Add/commit/push your changes to hello.c to the server.

  3. Use git status . to verify that you have no uncommitted files.

  4. Close your remote connection to the server (Click on “SSH: linux.cs.uchicago.edu” and choose the “Close remote connection” option or type Ctrl-D to logout).

  5. Exit VSCode.