Data Operations in UNIX
Computers are great at aiding in repetitive tasks. However, far too often, we forget that this applies just as much to our use of the computer as it does to the computations we want our programs to perform. We have a vast range of tools available at our fingertips that enable us to be more productive and solve more complex problems when working on any computer-related problem.
We want to teach you how to make the most of the tools you know, show you new tools to add to your toolbox, and hopefully instill in you some excitement for exploring (and perhaps building) more tools on your own.
Learning Outcomes
Navigate and operate UNIX-based command-line operations.
Chapter Learning Outcomes
- Use Unix commands such as
cp, scp, rm, mv
, including with options and arguments. - Use
grep
with options to answer file search questions. - Connect to a remote system using
ssh
and describe its purpose in secure communication - Use useful shortcuts and keyboard commands to manipulate the terminal more quickly.
- Troubleshoot errors when using Unix commands by interpreting terminal output and correcting syntax
Overview
A summary of how to access the terminal and how certain commands work is provided below for your reference. Read the individual sections to see more information about those commands.
Helpful Shortcuts in the Terminal
Up and Down arrows
Used to scroll through commands
Tab completion
Used for long file names
Chaining ;
Used for combining multiple commands
Example:
cd ~/ ; pwd ; ls
- The
clear
command. type in the wordclearand watch the screen adjust up so that you have a clean screen!
Smartphone Analogy
Voice Commands: Call my Dad’s Cell phone (Smartphone Analogy)
- Go to voice commands
- Find and use the right command
- Ex: Call
- Ex: Text (send a message?)
- Find the right person
- Ex: Dad
- Are there options?
- Ex: which number: cell (not required , used to provide extra information)
- Putting it all together: Call Dad’s Cell
Terminal Commands:
List the contents of ~/ with details
- Connect to terminal
- Find and use the right
command:
- Ex:
ls
- Find the right path
- Ex: ~/
- Are there options? • Ex: want details:
-l
(not required, used to provide
extra information) - Putting it all together:
ls -l ~/
Standard Syntax
Standard command line format:
eos$ command –options argument_1 argument_2
where…
eos$
is the prompt and is simply the marker for which system you are logged into. This is not part of the command line.command
is the command you wish to use–options
are used to modify the behavior of the commandargument_1
andargument_2
are the parameters of the command
VERY IMPORTANT…
eos$
is NEVER part of the syntax of a command. It is just the marker for the line, and if you are on a different machine, you’ll see some other line like%
. When asked a question about the syntax of a command,eos$
should NEVER be part of the answer.
Helpful Commands
Command | Purpose | Syntax |
---|---|---|
pwd |
Shows the absolute pathname of the user’s current directory | pwd |
ls |
List contents of a directory |
|
cd |
Change directory |
|
mkdir |
Create a new directory |
|
cp |
Copy a file or folder |
|
mv |
Move or rename a file |
|
rm |
Remove a file or directory |
|
clear |
Clear the terminal screen | clear |
man |
View manual page for a command |
|
script |
Start a terminal log file |
|
Log File: script
Command:script
- Options:
- -a : append. Allows you to append a log file
- -f : flushed. Writes to the file in real time. You should always use this option in E 115
- Note: To end a log file, you must type “exit”.
- The log file will create wherever you were when you executed the command.
- What is the output? Will tell you the file has started. Upon exit, will tell you the file is finished.
WHEN YOU ARE DONE: EXIT your log file and then always type logout to end your terminal session.
List the contents of a directory: ls
Command:ls
- Used to list the contents of a specific directory
- Options:
- -a : all. shows all files, including hidden ones
- -l : long. shows files in long format plus details
- Options can be used in any order or used independently of each other
- What is the output? the contents of the directory specified, OR of the current directory if nothing is specified
Examples
I want to list the contents of my HOME directory using a relative path name…
I am in my home directory, ~/
I am elsewhere in the file tree
ls
orls -a
(only if asked for all files)
orls -l
(only if asked for info on files)
orls -al
orls -la
(only if asked for all files with details)ls ~/
orls -a ~/
orls -l ~/
orls -al ~/
orls -la ~/
Make Directory: mkdir
Command:mkdir
- Used to create a new directory
- Options:
- -p : creates every directory in the pathname, assuming it doesn’t already exist
- If you are in the location, you can ignore that part of the syntax
- The terminal window is very literal, so make sure you either specify an exact location or be in that location.
Should you expect output? No
Advice: usepwd
or ls with or after using cd to ensure you’ve done it right.
I want to create a directory named MyE115 in my home directory…
I am in ~/
I am elsewhere in the file tree
mkdir MyE115
the terminal is CASE SENSITIVE
do an ls after to ensure proper creationmkdir ~/MyE115
the terminal is CASE SENSITIVE and there is NO SPACE between the ~/ and the name of the directory
do an ls ~/ after to ensure proper creation
Change directory: cd
Command:cd
- Used to change your current location
- Advice: use pwd or ls with or after using cd to ensure you’ve done it right.
Copy files: cp
Command:cp
- Used to copy a file from one location to another
- Also used to rename files and keeps the original
- Advice: use ls with or after using cd to ensure you’ve done it right.
I want to copy a file named example.txt from my HOME directory to my ~/MyE115 directory…
I am in~/
I am elsewhere in the file tree
cp example.txt MyE115
orcp example.txt MyE115/newName.txt
do an ls after to ensure proper copyingcp ~/example.txt ~/MyE115
orcp ~/example.txt ~/MyE115/newName.txt
do an ls after to ensure proper copying
Move files: mv
Command:mv
- Used to move a file from one location to another
- Also used to rename files and deletes the original
- Advice: use ls with or after using cd to ensure you’ve done it right.
- Cannot be used within the E 115 course locker. Students lack the permission for it.
I want to move a file named example.txt from my HOME directory to my ~/MyE115 directory…
I am in~/
I am elsewhere in the file tree
mv example.txt MyE115/newName.txt
do an ls after to ensure proper movingmv ~/example.txt ~/MyE115/newName.txt
do an ls after to ensure proper moving
Remove files or folders: rm
Command:rm
- Used to delete files or folders from the system
- Options:
- -r : recursive. Required for deleting directories
- -f : forced. deletes without asking permission
Should you expect output? Only if you don’t use -f
What is the output? asks if you want to remove the file or folder in question
Removing files and folders: an example
I am in ~/ and so is file.txt and folder
Files
rm file.txt
orrm -f file.txt
Folders
- rm -rf folder
*be careful when using -f!!
I am elsewhere in the file tree and file.txt and folder are in ~/
Files
rm ~/file.txt
orrm -f ~/file.txt
Folders
rm -rf ~/folder
*be careful when using -f!!
Manual pages: man
Command:man
- To scroll, use the arrow keys. To exit, press q
- Used for learning more information on given command
- What is the output? The contents of the manual page for the question
Exercises
Here are a few useful tools:
ping
: Test whether a computer is reachable on the network.curl
: Download a file or webpage from a URL.find
: Search for files by name, type, or modified time.
These aren’t required for this course, but you may use them in your other courses.