Introduction to /etc/passwd and /etc/shadow files in Linux systems (Debian/Ubuntu/CentOS/RHEL etc.)

Linux operating systems store all username and password (including administrators/root) in /etc/passwd and /etc/shadow file.

/etc/passwd

Each user has a line of corresponding record which records basic attributes. Only root/administrators can modify it. All other users have read only access to it.

/etc/shadow

As name suggested, this file is like shadow of “passwd” file. The record in “shadow” file is corresponding to the records in “passwd” file. Records is “shadow” file is automatically produced by “pwconv” command based on “passwd” file. Only root/administrators have read and write access to “shadow” file, other users can’t read it.

File permission for passwd and shadow
File permission for passwd and shadow

About /etc/passwd

sudo vi /etc/passwd
partial passwd file
partial passwd file
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin

There are 7 columns for each record

ColumnDescription
1Username
2Placeholder, x = password is required to login, empty = password is not required to login
3User UID
4User GID
5Extra information, Full name, contact information etc.
6Home directory
7Login shell, /bin/bash = Login to system shell enabled, /sbin/nologin = User can’t login

About /etc/shadow

sudo vi /etc/shadow
partial shadow file
partial shadow file
root:!:18313:0:99999:7:::
daemon:*:18313:0:99999:7:::
bin:*:18313:0:99999:7:::
sys:*:18313:0:99999:7:::
sync:*:18313:0:99999:7:::
games:*:18313:0:99999:7:::
man:*:18313:0:99999:7:::
lp:*:18313:0:99999:7:::
mail:*:18313:0:99999:7:::
news:*:18313:0:99999:7:::

There are 8 columns for each record

ColumnDecription
1Username
2Password (!! = no password, encrypted if password is set)
3Days between last change of password and 01/01/1970
4Minimum password age (Validated days)
5Maximum password age (Validated days)
6Buffer time (Days) after the password is expired (After the password is expired, for how many days the user can change the password, old password can’t be used to login again during this period of time)
7Number of days after password expires that account is disabled
8Date which the account is disabled (Days since 01/01/1979)
9Not used yet

Linux Command Line/ Terminal Disk Space Usage tool (Find largest folder/file)

For finding largest file/folder or showing disk space usage on Windows, refer to this one: How to: Find Largest file on Windows, Windows 7, Windows 10, Microsoft Windows, Windows Server (Disk Space Usage)

Ncdu (NCurses Disk Usage) is a command line tool to view and analyse disk space usage on Linux.

It can be easily installed on most Linux systems with package management system.

ncdu on Kali Linux 2020
ncdu on Kali Linux 2020

Debian/Kali Linux/Ubuntu etc. Linux installation

sudo apt install ncdu -y
 
OR
 
sudo aptitude ncdu -y

RHEL/CentOS/Fedora etc. Linux installation

If EPEL repo is not installed yet, we have to install EPEL repo first

sudo yum -y install epel-release

Next, we can now install ncdu

sudo yum install ncdu -y

Using ncdu is simple.

Show current working directory info

ncdu

Show info for a folder e.g. “/etc”

ncdu /etc

To show more info about a folder while in ncdu, press “i” key (Press “i” again to dismiss)

ncdu - i
ncdu – i

Press Shift + ? to show help document while in ncdu

ncdu help
ncdu help

Press “q” key to quit menus and the ncdu program


How to: Use shortcut keys/Key combinations in Linux Terminal

1 Tab

When entering command, enter beginning of the command, file name or folder name or command option then press “Tab” key, it will complete the rest for you automatically or show all possible results.

2 Ctrl + C

Terminate/Kill the command or process, it will terminate the running process immediately. (signal SIGINT). It can be intercepted by a program, thus the program can clean itself up before exiting or not exit at all.

3 Ctrl + Z

Suspending a process by sending the SIGSTOP signal, it cannot be intercepted by the program.

4 Ctrl + D

Exit the current terminal. If you are using SSH, it will close it. If you are using a terminal directly, it will close the terminal window.

5 Ctrl + L

Clear terminal screen, same effect as “clear” command

6 Ctrl + A

Move the type cursor to the beginning of the line (Same as pressing “Home” key on keyboard)

7 Ctrl + E

Move the type cursor to the end of the line (Same as pressing “End” key on keyboard)

8 Ctrl + U

Wipe the line and move the type cursor to the beginning of the line (Instead of use “Backspace” key to clear the line slowly)

9 Ctrl + K

Wipe the content from the type cursor to the end of the line

10 Ctrl + W

Clear a word

Before Ctrl + W

Before Ctrl + W
Before Ctrl + W

After Ctrl + W

After Ctrl + W
After Ctrl + W

11 Ctrl + Y

It will paste text removed by Ctrl + U, Ctrl + U and Ctrl + K. If you have deleted text by mistake, this will be helpful.

12 Ctrl + P

Review last command, use repetitively to go back further. Many Terminal provides this review function by PageUp key as well. Some provide the review function by using up arrow key as well (↑).

13 Ctrl + N

Similar usage as Ctrl + P but opposite direction, this command navigate to more recent commands. Many Terminal provides this review function by PageDown key as well. Some provide the review function by using down arrow key as well (↓).

14 Ctrl + R

Used for search history commands

Bonus:

Alternatively, we can use “history” command to show all history command

To search from history command, we can use “history | grep searchTerm”


Basics about Network configuration in Linux, IP commands, configuration files etc.

Table of Contents

1 Some useful basic IP commands

1.1 Use network/Interface configuration files to make permanent changes.

For CentOS/RHEL/Fedora etc.

File: /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE="eth0"
BOOTPROTO=static
ONBOOT=yes TYPE="Ethernet"
IPADDR=10.0.0.10
NAME="System eth0"
HWADDR=00:53:78:2C:7D:9E
GATEWAY=10.0.0.1

For Debian/Ubuntu/Kali Linux etc.

File: /etc/network/interfaces

auto eth0
iface eth0 inet static
address 10.0.0.10
netmask 255.255.255.0
gateway 10.0.0.1

Restart network services to make the changes take effect

sudo /etc/init.d/networking restart
 
OR
 
sudo service restart networking
 
OR
 
systemctl restart networking

1.2 Assign IP address to a specific interface (eth0 in this example) (nonpersistent, will be lost after system reboot)

sudo ip addr add 10.0.0.10 dev eth0

1.3 Remove IP address from a specific interface

sudo ip addr del 10.0.0.10/24 dev eth0

1.4 Check IP address

sudo ip addr
 
OR
 
sudo ip addr show
 
OR
 
sudo ifconfig

1.5 Enable Network interface

sudo ip link set eth0 up

1.6 Disable Network interface

sudo ip link set eth0 down

1.7 Check routing table

sudo ip route show

1.8 Add Static route

sudo ip route del 10.0.0.0/24

1.9 Add persistent static routes

For CentOS/RHEL/Fedora etc.

File: /etc/sysconfig/network-scripts/route-eth0

Add following

10.0.0.0/24 via 192.168.5.20 dev eth0

For Debian/Ubuntu/Kali Linux etc.

File: /etc/network/interfaces

Add following

up ip route add 10.0.0.0/24 via 192.168.5.20 dev eth0

Restart network services to make the changes take effect

sudo /etc/init.d/networking restart
 
OR
 
sudo service restart networking
 
OR
 
systemctl restart networking

1.10 Add default gateway

sudo ip route add default via 10.0.0.1

2 Network configuration file

For CentOS/RHEL/Fedora etc.

File: /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
   #Alias name for the NIC
BOOTPROTO={static|dhcp|none|bootp}
   #Boot protocol, static|none;dhcp
IPADDR=192.168.10.10
   #Set IP address
NETMASK=255.255.255.0
   #Netmask
GATEWAY=192.168.10.1
   #Gateway
ONBOOT=yes|no
   #Activate the network port or not, on boot
HWADDR=00:1E:0B:8F:B0:D0
   #MAC address, if same as the default MAC address of the hardware, this line can be omitted 
DNS1=202.106.0.20
   #Specifiy DNS server
USERCTL=yes|no
   #Users (non-admin/root) allowed to enable/disable this port or not
PEERDNS=yes|no
   #Accept/Reject the DNS server from DHCP while BOOTPROTO is dhcp

For Debian/Ubuntu/Kali Linux etc.

File: /etc/network/interfaces

auto eth1
     #Automatically connect to Ethernet on boot
iface eth1 inet static
     #Assign IP address by static/dhcp
address 192.168.72.8
     #IP address
netmask 255.255.255.0
     #Netmask
gateway 192.168.72.1
     #Default gateway
dns-nameservers 8.8.8.8 4.4.2.2
     #DNS server

3 Hosts configuration

File: /etc/hosts

192.168.0.10 internalserver.mynet

4 Network Interface Controller (NIC) Naming

lo: Localhost loop

ppp#: Point-to-Point Protocol

eth: Ethernet

5 Network management tool

network tool and NetworkManager tool.

network

Restart network

sudo /etc/init.d/network restart

NetworkManager

It can be used to manager network easily, when X Window is not available this GUI tool can be used to manage network without the need to edit configuration file manually.

nmtui
nmtui
nmtui
NetworkManager TUI - nmtui
NetworkManager TUI – mntui

6 NetworkManager cli

NetworkManager provides cli tools as well alongside nmtui

nmcli con show
     #Get UUID table
nmcli dev
     #Check network device status
nmcli r wifi off
     #Turn off wifi
nmcli - commands
nmcli – commands

Start NetworkManager on boot

chkconfig NetworkManager on
 
OR
 
 systemctl enable NetworkManager

Start NetworkManager immediately

service NetworkManager start
 
OR
 
 systemctl start NetworkManager

How to: Find which process is causing high CPU usage Linux/Ubutu/Debian/Kali Linux/CentOS/RHEL

“top” command

top shows CPU usage in real time.

By default, it lists process by their CPU usage, refreshes every 5 seconds.

We can use following command to show top 10 processes with highest CPU usage.

top -b | head -10
[email protected]:~# top -b | head -10
top - 02:05:40 up 20:24,  1 user,  load average: 0.00, 0.00, 0.00
Tasks: 165 total,   1 running, 164 sleeping,   0 stopped,   0 zombie
%Cpu(s):  3.1 us,  3.1 sy,  0.0 ni, 93.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   3913.3 total,   2066.4 free,    617.4 used,   1229.6 buff/cache
MiB Swap:   4094.0 total,   3850.2 free,    243.8 used.   3042.1 avail Mem 
    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
  61649 root      20   0    9144   3560   3164 R   6.7   0.1   0:00.01 top
      1 root      20   0  166420   5908   3936 S   0.0   0.1   0:04.54 systemd
      2 root      20   0       0      0      0 S   0.0   0.0   0:00.03 kthreadd
  • -b : Batch mode.
  • head -10: Display first 10 lines in the output.
  • PID : Unique ID of the process.
  • USER : Owner of the process.
  • PR : priority of the process.
  • NI : The NICE value of the process.
  • VIRT : How much virtual memory used by the process.
  • RES : How much physical memory used by the process.
  • SHR : How much shared memory used by the process.
  • S : This indicates the status of the process: S=sleep R=running Z=zombie.
  • %CPU : The percentage of CPU used by the process.
  • %MEM : The percentage of RAM used by the process.
  • TIME+ : How long the process being running.
  • COMMAND : Name of the process.

“ps” command

“ps” stands for “processes status”, it display the information about the active/running processes on the system.

We can use following command to find out high CPU usage processes.

ps -eo pid,ppid,%mem,%cpu,cmd --sort=-%cpu | head
[email protected]:~# ps -eo pid,ppid,%mem,%cpu,cmd --sort=-%cpu | head
    PID    PPID %MEM %CPU CMD
     78       2  0.0  0.4 [kswapd0]
    639     576  4.1  0.4 /usr/lib/xorg/Xorg :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch
    262       2  0.0  0.2 [kworker/1:1H-kblockd]
    267       2  0.0  0.1 [kworker/0:1H-kblockd]
    889     823  0.8  0.1 xfwm4 --display :0.0 --sm-client-id 24f144caf-a490-40f0-afc0-fd75665210e1
      1       0  0.1  0.0 /sbin/init splash
      2       0  0.0  0.0 [kthreadd]
      3       2  0.0  0.0 [rcu_gp]
      4       2  0.0  0.0 [rcu_par_gp]

To see the command name instead of full path.

ps -eo pid,ppid,%mem,%cpu,comm --sort=-%cpu | head
[email protected]:~# ps -eo pid,ppid,%mem,%cpu,comm --sort=-%cpu | head
    PID    PPID %MEM %CPU COMMAND
     78       2  0.0  0.4 kswapd0
    639     576  4.1  0.4 Xorg
    262       2  0.0  0.2 kworker/1:1H-kblockd
    267       2  0.0  0.1 kworker/0:1H-kblockd
    889     823  0.8  0.1 xfwm4
      1       0  0.1  0.0 systemd
      2       0  0.0  0.0 kthreadd
      3       2  0.0  0.0 rcu_gp
      4       2  0.0  0.0 rcu_par_gp
  • -e : Select all processes.
  • -o : To customize a output format.
  • –sort=-%cpu : Sort the ouput based on CPU usage.
  • head : To display first 10 lines of the output
  • PID : Unique ID of the process.
  • PPID : Unique ID of the parent process.
  • %MEM : The percentage of RAM used by the process.
  • %CPU : The percentage of CPU used by the process.
  • Command : Name of the process.

htop

htop is a command line utility that allows you to interactively monitor your system’s vital resources or server’s processes in real time

You might need to install htop first

#Debian/Ubuntu/Kali Linux etc.
sudo apt install htop
 
#CentOS/RHEL etc.
sudo yum install htop

To launch htop (Use “q” key to exit)

htop
htop
htop

We can easily sort the processes by their Priority, Nice, Virtual memory usage, Memory, CPU and running time etc. Simply by clicking on the column header.

glances

glances is another utility which can be easily installed and used system resource monitoring tool.

Install

#Debian/Ubuntu/Kali Linux etc.
sudo apt install glances
 
#CentOS/RHEL etc.
sudo yum install glances 

Launch (Use “q” key to exit)

glances

How to: Use “find” command in Linux (Debian, Ubuntu, Kali Linux, CentOS, RHEL/RedHat etc.)

Table of Contents

  • 1 Find command basics
  • 2 Find files and folders by their size
  • 3 Find files by their Owner/Group
  • 4 Find files and folders by date and time
  • 5 Find files by their privileges

1 Find command basics

1.1 Find file from current folder

(“Permission denied” error will appear if the current user doesn’t have permission to access that folder)

Find file named “1.txt” within current working folder

find filename
find 1.txt
find 1.txt
find 1.txt

1.2 Find the file from a folder

Find “1.txt” file under “/tmp” folder

find /path/to/folder -name filename
find /tmp -name 1.txt
find file from a parent folder
find file from a parent folder

1.3 Find file (Ignoring the case/Case insensitive)

Find “1.txt” within “/tmp” folder, ignoring case.

find /path/to/folder -iname filename
find /tmp -iname 1.txt
Find file (Ignoring the case/Case insencitive)
Find file (Ignoring the case/Case insencitive)

1.4 Find folder/directory only

find /path/to/folder -type d -name foldername
find /tmp -type d -name abc
Find folder/directory
Find folder/directory only
Find folder/directory, ignoring the case
Find folder/directory only, ignoring the case

1.5 Find file only

find /path/to/folder -type f -name 1
find /tmp -type f -name 1.txt
Find file only
Find file only

1.6 Find specific file

Find .txt files from “/tmp” folder

find -type f -name "*.extension"
find -type f -name "*.txt"
Find specific file
Find specific file

2 Find files and folders by their size

2.1 Find all 100MB files

find / -size 100M

2.2 Find files which is between 10MB and 100MB

find / -size +50M -size -100M

2.3 Find and delete files which is greater than 500MB in size

find / -size +500M -exec rm -rf {} \;

2.4 Find specific files and delete them

Find all mp3 files which are greater than 5MB, delete them

find / -type f -name *.mp3 -size +5M -exec rm {} \;

3 Find files by their Owner/Group

3.1 Find files by their owner

Find all of the “1.txt” file which belongs to Bob

find / -user Bob -name 1.txt

3.2 Find all files belongs to the user

Find all files belongs to Bob under “/test” folder

find /test -user Bob

3.3 Find files based on their group

Find all files under “/test” which belongs to the group “testgroup”

find /test -group testgroup

3.4 Find all “.txt” files under “/test” which belongs to Bob

find /test -user Bob -iname "*.txt"

4 Find files and folders by date and time

4.1 Find files

4.1 Find files which their data was last modified n*24 hours ago. (3 days in this example)

find / -mtime 3

4.2 Find files being accessed in 3 days

find / -atime 3

4.3 Find files that being modified between last 3-10 days

find / -mtime +3 -time -10

4.4 Find files which their status was last changed 5 minutes ago

find / -cmin -5

4.5 Find files which their data was last modified 5 minutes ago

find / -mmin -5

4.6 Find files which are being access within 5 minutes

find / -amin -5

5 Find files by their privileges

5.1 Find files with “777” permission

find -type f -perm 0777 -print

5.2 Find files without “777” permission

find / -type f ! -perm 777

5.3 Find files with “644” permission (SGID)

find / -perm 2644

5.4 Find files with “551” Sticky Bit permission

find / -perm 1551

5.5 Find SUID files

find / -perm /u=s

5.6 Find read only permission files

find / -perm u=r

5.7 Find executable files

find / -perm /a=x

5.8 Find all files with “777” permission and chmod to 644

find /test -type f -perm 0777 -print -exec chmod 644 {} \;

5.9 Find all folders with “777” permission and chmod to 755

find / -type d -perm 777 -print -exec chmod 755 {} \;

5.10 Find and delete single file (1.txt in this case)

find -type f -name "1.txt" -exec rm -f {} \;

5.11 Find and delete multiple files with same extension (txt files in this case)

find -type f -name "*.txt" -exec rm -f {} \;

5.12 Find all empty files

find / -type f -empty

5.13 Find all empty folders

find / -type d -empty

5.14 Find all hidden files under “/tmp” folder

find /tmp -type f -name ".*"

Quick Linux diagnostic commands (System Resource & Network)

System Resources

CPU:  uptime, top
RAM:  free, top
Disk: iostat, df -h, ds -S /home | sort -rn head -n 10

Network

NIC info:                   ethtool eth0
Interface info:             ifconfig eth0
routing info:               route -n
Internet Connection:        ping bing.com -c 5
DNS check 1:                dig bing.com
DNS check 2:                host bing.com
DNS check 3:                nslookup bing.com
WAN connection 1:           ping bing.com
WAN connection 2:           traceroute bing.com
Port Open/Close:            nmap -p 80 bing.com
Check host listening on:    netstat -tunlp | grep -w 80

Config Network

DNS:                            /etc/resolv.conf
Add default route:              route add default gw 10.0.0.1
Add route 192.168.0.0/24:       route add -net 192.168.0.0 netmask 255.255.0.0 dev eth0
Remove route 192.168.0.0/23:    route del -net 192.168.0.0 netmask 255.255.0.0 dev eth0
Restart network service 1:      /etc/init.d/networking restart
Restart network service 2:      service restart networking
Restart network service 3:      systemctl restart networking
Bring up NIC 1:                 ifup eth0
Bring up NIC 2:                 ip link set eth0 up
Stop NIC 1:                     ifdown eth0
Stop NIC 2:                     ip link set eth0 down
Host name 1:                    /etc/sysconfig/network
Host name 2:                    /etc/hosts