Linux – Basics, Useful Terminal commands and Basic File manipulation, (QuickStart)

Last Updated on

Basics/ Useful Terminal commands

Home Directory

Where most of the time the user files are stored.

Usually, the home directory is stored under “/home/” e.g. If we have created a user named “Bob” his home directory will be “/home/bob/” Bob will have full control over this folder including Create, Retrieve, Update, Delete (CRUD).

The corresponding folder for Microsoft Windows is usually “C:\Users\bob\”

Check current working directory

Which directory am I working in?

pwd
pwd
pwd

Switch directory/Directory navigation

Switch working directory, navigating through different directory/folders

e.g. Current working directory is “/tmp”, we want to navigate to “/tmp/test/”

cd test
cd
cd

Or using the absolute path

cd /tmp/test

List files and folders under a folder

List files and folders under current directory

ls
ls
ls

list files and folders under specific path

ls /tmp/test
ls /tmp/test
ls /tmp/test

Color scheme for “ls” command

BlueDirectory/Folder
RedArchive files
WhiteText file

Create a new text file

Create a “b.txt” text file under current working directory. File extension like “txt” is not necessary, but it’s easier for users when the extension is present.

touch b.txt
touch b.txt
touch b.txt

Create a new folder

mkdir newFolder

We can also use standard stream redirection to create file and write content to the text file.

ls > ls.txt
ls > ls.txt
ls > ls.txt

The output of “ls” command will be redirected and written to the “ls.txt” file

Copy, Delete, Move/Rename

Copy

cp ls.txt ls1.txt
cp
cp

Delete

rm ls1.txt
rm ls1.txt
rm ls1.txt
  • -f: –force ignore nonexistent files and arguments, never prompt
  • -r, -R: –recursive remove directories and their contents recursively
  • /*: Anything under root directory (Warning: Use carefully and do not try /* without full backup)
#Delete trash folder and everything within it
rm -rf trash

Rename

mv ls.txt ls1.txt

Move

#Under same working directory/folder
mv ls.txt ls1.txt
#With full path
mv /tmp/test/ls.txt /tmp/test/ls1.txt 
#Move to another folder with same file name
mv /tmp/test/ls.txt /tmp/ls.txt
#Move to another folder with different file name
mv /tmp/test/ls.txt /tmp/new.txt

Search/Find file

Find

Search from root level with “find”

find / -name abc.txt
find / -name abc.txt
find / -name abc.txt

Search from other specific path with “find”

find /tmp/test -name abc.txt

locate

Search with “locate”

updatedb
locate abc.txt
locate abc.txt
locate abc.txt

Search from other specific path with “locate”

updatedb
locate "/bob/*abc.txt"
locate "/root/*.abc.txt"
locate “/root/*.abc.txt”

“find” is usually slower than “locate”

It’s best to do “updatedb” before using “locate”

Search text in file

grep

Search text file (case-sensitive by default)

grep TextWeWantToSearch /tmp/test/textfile.txt

Search test file (ignore case)

grep -i TextWeWantToSearch /tmp/test/textfile.txt
grep, search text from file
grep, search text from file

Search all “txt” files under same directory

grep SearchCriteria /tmp/test/*.txt
grep, search text from any txt file under folder
grep, search text from any txt file under folder

Add “-n” switch to show line numbers

grep -n SearchCriteria /tmp/test/*.txt
grep with line numbers
grep with line numbers

If we want to search sub-directories as well, we need to add “-R” (recursive) parameter

grep -nR SearchCriteria /tmp/test/*
grep including sub-directories and any file with line number
grep including sub-directories and any file with line number

Autocomplete

When typing command, e.g. “fin” then hit “Tab” key twice.

Autocomplete
Autocomplete

If we continue to type “c” at the end, then hit “Enter”, “finecore” will automatically completed.

Same technique can be used for folder names as well. If there is only one folder start with “mus” then after we hit “Tab” key, music will be automatically completed, If there is multiple folder stars with “mus”, then hit “Tab” key twice, options will appear just like for other commands.

Basic File manipulation

Check metadata (Privilege, Size etc.)

ls -l
ls -l
ls -l

Basically the layout has 7 sections

File permissions + No. of hard links + File owner + Group the file belongs + File Size + File modification time + File Name

The 1st character represent file type

dFolder/Directory
File
sSocket
lLink

etc.

ls, output explained
ls, output explained

Following 9 characters:

r: Read, w: Write, x: Execute -: No privilege

The following group of three characters represent owner’s permission, next group of three characters represent the permission for the group which the user belongs to, last three characters represent privilege for others who is not owner and not the group the file belongs to.

Add “-h” switch together with “-l” to list files and folders with human readable size format

ls -hl
ls -hl
ls -hl

Add “-a” switch to show all files and folders including hidden ones

ls -al

We can also use stat to find detailed information about a file/folder/directory

stat filename

Output

[email protected]:/# stat home
   File: home
   Size: 4096            Blocks: 8          IO Block: 4096   directory
 Device: 801h/2049d      Inode: 5242881     Links: 2
 Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
 Access: 2020-01-05 15:31:07.026808761 +1100
 Modify: 2019-11-09 20:18:26.000000000 +1100
 Change: 2019-12-01 13:40:26.043449434 +1100
  Birth: -

Change/Modify file permission

Extended reading: Unix/Linux (Ubuntu, Debian, Kali Linux etc.) Privilege Management

Method 1: “chmod” with characters

+: Add, -: Remove, =: set permission.

a: all, o: others, u:user/owner,g:group

Set Read,Write,Execute rights for the file owner/user

chmod u=rwx a.txt

Add write permission for others

chmod o+w a.txt

Remove read permission from group

chmod g-r a.txt

Add execute permission for all users

chmod a+x a.sh

If we do not define which user/group etc to set the permission for. It will change the permission for all, including owner, group, others.

chmod +x a.sh

Method 2: “chmod” with numbers

4: r, 2: w, 1: x

They adds up in group of three.

Following command gives owner, group and others: rwx, rw, r permission. 7 = sum of r,w,x. 6 = sum of r, w. 1 = x.

chmod 761 a.txt

Change/Modify Access time/Modify time

Change access and modify time to 2001-01-01 00:00:01.000000000

touch -t 200101010000.01 file
touch -t 200101010000.01 a
touch -t 200101010000.01 a

Other ways to define date/time

touch -d "2001-01-01 00:00:01" -a file
 
touch -d "5 hours ago" -a file
 
touch -d "next monday" -m file

-d: Human readable date format

-a: access time

-m: modification time

Find out File type

file FileName
file /folder/*
file *
file *
file FileName
file FileName

Can’t run downloaded Program/Execute script etc.

We need to give the program or the script execute permission for it to run.

We can use “chmod” command which we have mentioned before.

Folder/Directory/File disk usage

#All file and folders under this folder
du -h *
du -h *, ls-hl
du -h *, ls-hl

-h switch represents human readable unit

Note: “du” outputs size occupied on the disk not the actual file size, whereas “ls -hl” outputs file size.

du -h a.png
du -h a.png
a.png Properties in Microsoft Windows
a.png Properties in Microsoft Windows

-s: Show total size of all files and sub-folders

-S: Show size of individual folders

du -hs
du -hS
du -hs, du -hS
du -hs, du -hS

Use “df” to show size of system folders

df
df -h
df -h, df
df -h, df

Compress/Extract file/Archive file

Deal with .tar, .tar.gz, .bz2 files

Create archive (.tar)

tar -cvf archive.tar folderName/

-c: Create archive

-v: Show files added to tarball

-f: Specify file name

Extract archive

tar -xvf archive.tar

-x: Extract

(-C: To extract to other location)

Create archive (.tar.gz, .tar.bz2)

To create “.tar.gz” use “-z”

tar -zcvf archive.tar.gz folderName/

To create “tar.bz2” use “-j”

tar -jcvf archive.tar.bz2 folderName/

To list content within archive file

tar -tvf archive.tar
tar -tvf archive.tar.gz
tar -tvf archive.tar.bz2

To extract “.tar.gz” and “tar.bz2”

tar -zxvf archive.tar.gz
tar -jxvf archive.tarb.z2

Extract to specific directory/folder

tar -zxvf archive.tar.gz --directory /folder/path
tar -jxvf archive.tarb.z2 --directory /folder/path

Parameters

  • x : Extract files
  • f : Tar archive name
  • –directory : Set directory name to extract files
  • -C : Set dir name to extract files
  • -j : Work on .tar.gz file format
  • -z : Work on .tar.bz2 file format
  • -v : Verbose output i.e. show progress on screen

Create Zip file

zip -r archive.zip folder
zip -r archive.zip folder1 folder2
zip archive.zip /path/to/folder1 /path/to/folder2

Extract Zip file

#Extract archive.zip to tmp folder
unzip archive.zip -d /tmp

View and Save output from command at same time, Output to two files at same time

We will use “tee” command

#echo "test" on screen and output to "1.txt" file
echo "test" | tee 1.txt
#echo "test" on screen and output to file "1.txt" and "2.txt"
echo "test" | tee 1.txt > 2.txt

Redirection/Pipeline

System Input/Output

Default Input:Standard InputSTDIN0
Default Output:Standard OutputSTDOUT1
Standard error outputSTDERR22
Standard Input:Keyboard
Standard OutputMonitor/Display
Standard Error OutputMonitor/Display

I/O redirection in Linux

TypeCodeRedirection
stdin0< or <<
stdout1> or >>
stderr22> or 2>>

“>”: Will Overwrite the file

“>>”: Will Append to the file

“set -C”: Disable redirection overwrite. (To force to overwrite use “>|”)

“set +C”: To disable “set -C”

“2>”: Redirect error output

“2>>”: Append mode

Write output and error to different location/file

ls / /aaa1 > /tmp/right1.txt 2> /tmp/err1.txt
ls /aaa1 /tmp/right2.txt 2> /tmp/err2.txt

“/dev/null”: Blackhole, redirect anything to it, nothing will be returned

“&>:”: Redirect standard output and error output to same file (or “2>&1”)

“<“: Use file as input rather than keyboard

echo 'Test message' > output
cat < output
tr 'a-z' 'A-Z' < /etc/fstab

“<<“: End of the file (Here document)

# cat < EOF
> hello
> world
> EOF
'# cat < EOF'
‘# cat < EOF’

“<<<“: The string following the “<<<” becomes the content of the “here” document.

cat <<<'hello world'

“>&2”: Redirect to a specific file descriptor

echo 'Error: Something went wrong' >&2

Pipeline

Use the output from previous command as input for next comand

Command 1 | Command 2 | …

# echo 'Test' | tr 'a-z 'A-Z'
# ls / | tr 'a-z 'A-Z'
# echo 'Test Message!' | tee /tmp/testf.txt

“tee”: Display the content and save to file

Added on 08/02/2020

wget

wget is included in most Linux distros.

We can use wget to download files

wget http://contoso.com/image.jpg

scp

scp – Secure Copy

We can use this command to download/upload files to/from remote server.

e.g. Download file from remote server

scp [email protected]: /path/to/test.txt /path/to/local/directory

e.g. Upload file to remote server

scp /path/to/local/folder [email protected]: /path/to/destination/folder

with “-r” switch, we can download/upload folder

ssh-keygen

Used to create SSH keypair which can be used in Bitbucket, Gitlab etc. to establish secure connection.

ssh-keygen -t ed25519

Combine commands

We can use ; to execute multiple commands without worrying about the previous command succeeded or failed.

ls -l; pwd

Execute second command only if the first command succeeded

mkdir newFolder && cd newFolder

Execute second command only if the first command failed

ping bing.com -c 1 || echo 'Ping bing.com failed'

Leave a Reply

Your email address will not be published. Required fields are marked *