Introduction to /etc/passwd and /etc/shadow files in Linux systems (Debian/Ubuntu/CentOS/RHEL etc.)

Linux operating systems store all username and password (including administrators/root) in /etc/passwd and /etc/shadow file.

/etc/passwd

Each user has a line of corresponding record which records basic attributes. Only root/administrators can modify it. All other users have read only access to it.

/etc/shadow

As name suggested, this file is like shadow of “passwd” file. The record in “shadow” file is corresponding to the records in “passwd” file. Records is “shadow” file is automatically produced by “pwconv” command based on “passwd” file. Only root/administrators have read and write access to “shadow” file, other users can’t read it.

File permission for passwd and shadow
File permission for passwd and shadow

About /etc/passwd

sudo vi /etc/passwd
partial passwd file
partial passwd file
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin

There are 7 columns for each record

ColumnDescription
1Username
2Placeholder, x = password is required to login, empty = password is not required to login
3User UID
4User GID
5Extra information, Full name, contact information etc.
6Home directory
7Login shell, /bin/bash = Login to system shell enabled, /sbin/nologin = User can’t login

About /etc/shadow

sudo vi /etc/shadow
partial shadow file
partial shadow file
root:!:18313:0:99999:7:::
daemon:*:18313:0:99999:7:::
bin:*:18313:0:99999:7:::
sys:*:18313:0:99999:7:::
sync:*:18313:0:99999:7:::
games:*:18313:0:99999:7:::
man:*:18313:0:99999:7:::
lp:*:18313:0:99999:7:::
mail:*:18313:0:99999:7:::
news:*:18313:0:99999:7:::

There are 8 columns for each record

ColumnDecription
1Username
2Password (!! = no password, encrypted if password is set)
3Days between last change of password and 01/01/1970
4Minimum password age (Validated days)
5Maximum password age (Validated days)
6Buffer time (Days) after the password is expired (After the password is expired, for how many days the user can change the password, old password can’t be used to login again during this period of time)
7Number of days after password expires that account is disabled
8Date which the account is disabled (Days since 01/01/1979)
9Not used yet

How to: Run Linux commands with time limit/timeout (Kill process/command after some time)

Sometimes we want to stop or kill the command after a period of time, so that we don’t get stuck with that command and wasting resources etc. To specify timeout or time limit for Linux command, we can use timeout command

Command Usage/Parameters

timeout [OPTION] DURATION COMMAND [ARG]...

DURATION is integer or floating point with unit

s: Seconds (Default)

m: Minutes

h: Hours

d: Days

Without units appended, by default it is considered as seconds.

If the DURATION is 0, the timeout is disabled.

Basic Usage

Timeout ping command after 3 seconds

timeout 3 ping 127.0.0.1
timeout 3 ping 127.0.0.1
timeout 3 ping 127.0.0.1

Timeout ping command after 3 minutes

timeout 3m ping 127.0.0.1

Timeout ping command after 3 days

timeout 1d ping 127.0.0.1

Timeout ping command after 3.2 seconds

timeout 3.2s ping 127.0.0.1

Send specific signal after timeout

By default if signal is not specified, timeout command will use “SIGTERM” signal after timeout. We can use -s (-signal) switch to specific which signal to send after timeout

e.g. Send SIGKILL signal to ping command after 3 seconds

sudo timeout -s SIGKILL 3s ping 127.0.0.1
sudo timeout -s SIGKILL 3s ping 127.0.0.1
sudo timeout -s SIGKILL 3s ping 127.0.0.1

We can use the name of the signal or the number of the signal

e.g. We can use 9 as SIGKILL to achieve same result

sudo timeout -s 9 3s ping 127.0.0.1
sudo timeout -s 9 3s ping 127.0.0.1
sudo timeout -s 9 3s ping 127.0.0.1

To list all acceptable signal, we can use kill -l to find out

kill -l
[email protected]:~# kill -l
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL       5) SIGTRAP
 6) SIGABRT      7) SIGBUS       8) SIGFPE       9) SIGKILL     10) SIGUSR1
11) SIGSEGV     12) SIGUSR2     13) SIGPIPE     14) SIGALRM     15) SIGTERM
16) SIGSTKFLT   17) SIGCHLD     18) SIGCONT     19) SIGSTOP     20) SIGTSTP
21) SIGTTIN     22) SIGTTOU     23) SIGURG      24) SIGXCPU     25) SIGXFSZ
26) SIGVTALRM   27) SIGPROF     28) SIGWINCH    29) SIGIO       30) SIGPWR
31) SIGSYS      34) SIGRTMIN    35) SIGRTMIN+1  36) SIGRTMIN+2  37) SIGRTMIN+3
38) SIGRTMIN+4  39) SIGRTMIN+5  40) SIGRTMIN+6  41) SIGRTMIN+7  42) SIGRTMIN+8
43) SIGRTMIN+9  44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13
48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12
53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9  56) SIGRTMAX-8  57) SIGRTMAX-7
58) SIGRTMAX-6  59) SIGRTMAX-5  60) SIGRTMAX-4  61) SIGRTMAX-3  62) SIGRTMAX-2
63) SIGRTMAX-1  64) SIGRTMAX
kill -l
kill -l

Stop frozen process

SIGTERM, the default signal can be ignored by some processes, thus the program will keep running. To make sure the process is killed, we can use -k (–kill after) switch with specified time. When the time limited reached, force to kill the process.

e.g. Let the shell script run for 2 minutes, if it did not exit, then kill after 5 seconds

timeout -k 5s 2m sh test.sh

By default the timeout command will run in background, if we want to run it in foreground, refer to following example

timeout --foreground 2m ./test.sh

timeout help

Usage: timeout [OPTION] DURATION COMMAND [ARG]...
  or:  timeout [OPTION]
Start COMMAND, and kill it if still running after DURATION.
Mandatory arguments to long options are mandatory for short options too.
      --preserve-status
                 exit with the same status as COMMAND, even when the
                   command times out
      --foreground
                 when not running timeout directly from a shell prompt,
                   allow COMMAND to read from the TTY and get TTY signals;
                   in this mode, children of COMMAND will not be timed out
  -k, --kill-after=DURATION
                 also send a KILL signal if COMMAND is still running
                   this long after the initial signal was sent
  -s, --signal=SIGNAL
                 specify the signal to be sent on timeout;
                   SIGNAL may be a name like 'HUP' or a number;
                   see 'kill -l' for a list of signals
  -v, --verbose  diagnose to stderr any signal sent upon timeout
      --help     display this help and exit
      --version  output version information and exit
DURATION is a floating point number with an optional suffix:
's' for seconds (the default), 'm' for minutes, 'h' for hours or 'd' for days.
A duration of 0 disables the associated timeout.
If the command times out, and --preserve-status is not set, then exit with
status 124.  Otherwise, exit with the status of COMMAND.  If no signal
is specified, send the TERM signal upon timeout.  The TERM signal kills
any process that does not block or catch that signal.  It may be necessary
to use the KILL (9) signal, since this signal cannot be caught, in which
case the exit status is 128+9 rather than 124.
GNU coreutils online help: <https://www.gnu.org/software/coreutils/>
Full documentation at: <https://www.gnu.org/software/coreutils/timeout>
or available locally via: info '(coreutils) timeout invocation'

How to: Use curl to submit Google Form

Before crafting the curl command, we need to find the “name” attributes for the text boxes and the value for them, we can use built-in developer tools from different browsers to find that out, record the value of the “name”

Now we can craft the curl command

curl https://docs.google.com/forms/d/e/stringoForTheGoogleForm/formResponse -d ifq -d "entry.1773835858=text1" -d "entry.784619947=text2" -d submit=Submit

Make sure after the string from copied URL we put “/formResponse”, it there is “viewform” at the end of the URL replace that with “formResponse”


“entry.1773835858” and “entry.784619947” are the “name” value for two text boxes

Now we can use the above curl command to submit the Google Form


Linux Command Line/ Terminal Disk Space Usage tool (Find largest folder/file)

For finding largest file/folder or showing disk space usage on Windows, refer to this one: How to: Find Largest file on Windows, Windows 7, Windows 10, Microsoft Windows, Windows Server (Disk Space Usage)

Ncdu (NCurses Disk Usage) is a command line tool to view and analyse disk space usage on Linux.

It can be easily installed on most Linux systems with package management system.

ncdu on Kali Linux 2020
ncdu on Kali Linux 2020

Debian/Kali Linux/Ubuntu etc. Linux installation

sudo apt install ncdu -y
 
OR
 
sudo aptitude ncdu -y

RHEL/CentOS/Fedora etc. Linux installation

If EPEL repo is not installed yet, we have to install EPEL repo first

sudo yum -y install epel-release

Next, we can now install ncdu

sudo yum install ncdu -y

Using ncdu is simple.

Show current working directory info

ncdu

Show info for a folder e.g. “/etc”

ncdu /etc

To show more info about a folder while in ncdu, press “i” key (Press “i” again to dismiss)

ncdu - i
ncdu – i

Press Shift + ? to show help document while in ncdu

ncdu help
ncdu help

Press “q” key to quit menus and the ncdu program


How to: Use shortcut keys/Key combinations in Linux Terminal

1 Tab

When entering command, enter beginning of the command, file name or folder name or command option then press “Tab” key, it will complete the rest for you automatically or show all possible results.

2 Ctrl + C

Terminate/Kill the command or process, it will terminate the running process immediately. (signal SIGINT). It can be intercepted by a program, thus the program can clean itself up before exiting or not exit at all.

3 Ctrl + Z

Suspending a process by sending the SIGSTOP signal, it cannot be intercepted by the program.

4 Ctrl + D

Exit the current terminal. If you are using SSH, it will close it. If you are using a terminal directly, it will close the terminal window.

5 Ctrl + L

Clear terminal screen, same effect as “clear” command

6 Ctrl + A

Move the type cursor to the beginning of the line (Same as pressing “Home” key on keyboard)

7 Ctrl + E

Move the type cursor to the end of the line (Same as pressing “End” key on keyboard)

8 Ctrl + U

Wipe the line and move the type cursor to the beginning of the line (Instead of use “Backspace” key to clear the line slowly)

9 Ctrl + K

Wipe the content from the type cursor to the end of the line

10 Ctrl + W

Clear a word

Before Ctrl + W

Before Ctrl + W
Before Ctrl + W

After Ctrl + W

After Ctrl + W
After Ctrl + W

11 Ctrl + Y

It will paste text removed by Ctrl + U, Ctrl + U and Ctrl + K. If you have deleted text by mistake, this will be helpful.

12 Ctrl + P

Review last command, use repetitively to go back further. Many Terminal provides this review function by PageUp key as well. Some provide the review function by using up arrow key as well (↑).

13 Ctrl + N

Similar usage as Ctrl + P but opposite direction, this command navigate to more recent commands. Many Terminal provides this review function by PageDown key as well. Some provide the review function by using down arrow key as well (↓).

14 Ctrl + R

Used for search history commands

Bonus:

Alternatively, we can use “history” command to show all history command

To search from history command, we can use “history | grep searchTerm”


How to: Install Xrdp (Remote Desktop) on CentOS 8 – (How to: Use Windows to remote control CentOS 8)

Xrdp is an open-source implementation of the Microsoft Remote Desktop Protocol (RDP) that allows you to graphically control a remote system

1 If you don’t have GUI/Desktop Environment installed on your CentOS 8, install it by using following command

sudo dnf groupinstall "Server with GUI"

2 Install EPEL repo stands for Extra Packages for Enterprise Linux Repository

sudo dnf -y install epel-release

3 Install xrdp

sudo dnf -y install xrdp

4 (To enable the service on boot and start it now)

sudo systemctl enable xrdp --now

5 (Just start the service without enable it on boot)

sudo systemctl start xrdp

6 Check status of xrdp

sudo systemctl status xrdp

7 Configure Xrdp

Add

exec gnome-session

to the end of the configuration file

/etc/xrdp/xrdp.ini

Or use following command to add “exec gnome-session” to the end of the “/etc/xrdp/xrdp.ini” configuration file

sudo bash -c 'echo "exec gnome-session" >> /etc/xrdp/xrdp.ini'

8 Restart xrdp to load the new configuration

sudo systemctl restart xrdp

9 Configure firewall to allow incoming connection to xrdp

(To allow incoming connection from anywhere to port 3389)

sudo firewall-cmd --add-port=3389/tcp --permanent
sudo firewall-cmd --reload

(To only allow incoming connection from LAN e.g. 10.0.0.0/24 to 3389 port)

sudo firewall-cmd --new-zone=xrdp --permanent
sudo firewall-cmd --zone=xrdp --add-port=3389/tcp --permanent
sudo firewall-cmd --zone=xrdp --add-source=10.0.0.0/24 --permanent
sudo firewall-cmd --reload

Note: More secure way to do is to USE VPN then connect to xrdp server or only accept incoming connection to port 3389 from localhost then setup SSH tunnel to securely forward traffic from your local server on port 3389 to the server on the same port.

Now we use Windows Remote Desktop Connection to connect to the CentOS 8 via xrdp.


Kali Linux – pip/pip3 install -r requirements.txt fail

The Error

When executing following command in Kali Linux

pip install -r requirements.txt
 
OR
 
pip3 install -r requirements.txt

We get error

Command “python setup.py egg_info” failed with error code 1 in ….

Or other errors

We can give following fix a try

The Fix

Use following command instead

pip install --upgrade --force-reinstall -r requirements.txt
 
OR
 
pip3 install --upgrade --force-reinstall -r requirements.txt 

What’s the difference between CentOS Linux vs CentOS Stream

1 If you just want to test CentOS or use it as server: Download CentOS Linux

2 If you want to know what’s difference, read on

In short, CentOS Stream is in-between Fedora and RHEL, it provides clearner vision of what the next version of RHEL will be, give developer time to plan and create next-generation applications, so that they are compatible with future RHEL version. Whereas the CentOS Linux is the normal version we should use if we just want to setup another server.

Resources