How to: Check open ports quickly in Linux

Nmap

sudo nmap -sT -p- 10.0.0.1

-sT: Scan TCP ports

-p-: All 65535 ports

(Without -p-, by default Nmap will scan 1000 ports)

Output

Starting Nmap 7.80 ( https://nmap.org ) at 2020-03-22 23:10 CEST
Nmap scan report for 10.0.0.1
Host is up (0.00010s latency).
Not shown: 998 closed ports
PORT   STATE SERVICE
22/tcp open  ssh
80/tcp open  http
MAC Address: xx:xx:xx:xx:xx:xx
Nmap done: 1 IP address (1 host up) scanned in 0.50 seconds

The above scan shows that only ports 22, 80 are open.

To scan UDP ports use -sU instead of -sT

sudo nmap -sU -p- 10.0.0.1

Netcat

nc -zv 10.0.0.1 20-443

-z: Only scan for open ports without sending any data

-v: Set verbosity level (can be used several times)

Scan host 10.0.0.1 for open ports from 20 to 443

Output

nc: connect to 10.0.0.1 port 20 (tcp) failed: Connection refused
nc: connect to 10.0.0.1 port 21 (tcp) failed: Connection refused
Connection to 10.0.0.1 22 port [tcp/ssh] succeeded!
…
Connection to 10.0.0.1 80 port [tcp/http] succeeded!
Connection to 10.0.0.1 443 port [tcp/https] succeeded!

-u: Scan UDP ports

nc -z -v -u 10.0.0.1 20-443

Only display open ports with grep

nc -z -v 10.0.0.1 20-80 2>&1 | grep succeeded
 
# Output
Connection to 10.0.0.1 22 port [tcp/ssh] succeeded!
Connection to 10.0.0.1 80 port [tcp/http] succeeded!
Connection to 10.0.0.1 443 port [tcp/https] succeeded!

How to: Create shortcut icon for Anaconda/Anaconda3 Navigator/Launch Anaconda Navigator in Linux/Debian/Ubuntu/Kali Linux (How to create desktop shortcuts in Linux)

(For how to install Anaconda and Anaconda Navigator: How to: Install Anaconda/Anaconda3 in Linux/Debian/Ubuntu/Kali Linux)

To Launch Anaconda Navigator via command line

If Anaconda is installed correctly. We should be able to launch anaconda navigator by using following command

anaconda-navigator
Launch anaconda-navigator
Launch anaconda-navigator

If the above command doesn’t work we need to add the anaconda3 path to environment, we can use following command (Replace “yourusername” to your real system username)

export PATH=/home/yourusername/anaconda3/bin:$PATH

(e.g. export PATH=/home/abc/anaconda3/bin:$PATH)

Now we should be able to use “anaconda-navigator” command to launch Anaconda Navigator.

To Launch Anaconda Navigator via Desktop shortcut/icon

We need to manually create desktop shortcut icon first

1 Create an empty file on desktop name it anaconda-navigator.desktop

2 Open the file and enter following contents

(Replace “abc” with your system username, or replace whole path if you have installed to other location rather than home directory)

#!/usr/bin/env xdg-open
[Desktop Entry]
Name=Anaconda
Version=2.0
Type=Application
Exec=/home/abc/anaconda3/bin/anaconda-navigator
Icon=/home/abc/anaconda3/lib/python3.7/site-packages/anaconda_navigator/static/images/anaconda-icon-256x256.png
Comment=Open Anaconda Navigator
Terminal=false
#!/usr/bin/env xdg-open
[Desktop Entry]
Name=Anaconda
Version=2.0
Type=Application
Exec=/path/to/anaconda3/bin/anaconda-navigator
Icon=/path/to/anaconda3/lib/pythonVersion/site-packages/anaconda_navigator/static/images/anaconda-icon-256x256.png
Comment=Open Anaconda Navigator
Terminal=false
Anaconda desktop icon/shortcut
Anaconda desktop icon/shortcut

3 Although we can use the shortcut on desktop, it will not be listed in application screen/menu, we can use following command to make it appear in our application screen/menu as well

cp anaconda-navigator.desktop ~/.local/share/applications/
Copy shortcut to application menu/list
copy shortcut to application menu/list

4 Now it will appear in our application menu/list

Application menu/list
Application menu/list

Resource

Anaconda-Navigator – Ubuntu16.04


How to: Monitor CPU/GPU/HDD temperature in Linux (Debian/Ubuntu/Kali Linux/CentOS/RHEL etc.) easily

1 glances

Monitors CPU usage, RAM usage, SWAP usage, system load, process list, hard drive I/O, Network I/O, sensors (temperature), battery, file system usage, Docker, Monitor, alarm, system information, up time etc.

Install on Debian/Ubuntu/Kali Linux

sudo apt install glances -y

Usage

sudo glances
glances
glances

2 sensor

By default, it’s installed on some Linux distros like Ubuntu etc. It can show CPU, HDD, temperature, fan RPM etc.

Install on Debian/Ubuntu/Kali Linux

sudo apt install lm-sensors

Usage

# Initialize/Detect sensors
sudo sensors-detect
 
# Show sensors information
sudo sensors

Output

coretemp-isa-0000 Adapter: ISA adapter Core 0:       +47.0°C  (high = +105.0°C, crit = +105.0°C) Core 1:       +46.0°C  (high = +105.0°C, crit = +105.0°C) acpitz-virtual-0 Adapter: Virtual device temp1:        +51.0°C  (crit = +105.0°C) thinkpad-isa-0000 Adapter: ISA adapter fan1:         674 RPM temp1:        +50.0°C temp2:        +51.0°C temp3:        +52.0°C temp4:         -1.0°C temp5:         +0.0°C temp6:         +0.0°C temp7:        +32.0°C temp8:         +0.0°C

How to: Start/Use/Initialize OpenVAS – Open Vulnerability Assessment Scanner on Kali Linux (Intro)

Before using the OpenVAS, we need to setup and update it.

1 Launch a terminal, and run setup for OpenVAS

sudo openvas-setup

Wait until it finishes downloading and updating, it will take awhile

2 When it’s done, it will show the admin login username and admin login password, note them down, we will need them every time we try to login to OpenVAS

openvas-setup done
openvas-setup done

*3 Update feed for OpenVAS (Only required if there is new updates), when initializing, this step was done once already.

sudo openvas-feed-update

If failed (You might encounter this error)

rsync: failed to connect to feed.openvas.org (xx.xx.xx.xx): Connection refused (111)
rsync: failed to connect to feed.openvas.org (xx:xx:xx:xx::xx): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(127) [Receiver=3.1.3]

Just try again with the same command, it should get through.

4 Launch OpenVAS

sudo openvas-start

It will tell us the address for webui, in this case, it is https://127.0.0.1:9392

OpenVAS webui
OpenVAS webui

(We might encounter following error)

Failed to execute default Web Browser
Failed to execute default Web Browser

It’s OK, just close it, then launch our favourite web browser then enter https://127.0.0.1:9392 as the address

Now we should have the OpenVAS login screen in front of us.

OpenVAS login screen
OpenVAS login screen

5 Enter your login detail recorded from step 2

Now you will see the Dashboard of OpenVAS.

Happy hunting/fixing 🙂


How to: Install Anaconda/Anaconda3 in Linux/Debian/Ubuntu/Kali Linux

(For how to launch Anaconda Navigator or create Anaconda Navigator shortcut: How to: Create shortcut icon for Anaconda/Anaconda3 Navigator/Launch Anaconda Navigator in Linux/Debian/Ubuntu/Kali Linux)

1 Download Anaconda3 from official website: https://www.anaconda.com/distribution/

https://repo.anaconda.com/archive/Anaconda3-2019.10-Linux-x86_64.sh

2 Make it executable

2.1 We can right click on the file, then click on “Properties” -> “Permissions” -> Check “Allow this file to run as a program”

Assign execute permission
Assign execute permission

2.2 We can also use “chomd” command to assign execute permission

Related to chmod command: Unix/Linux (Ubuntu, Debian, Kali Linux etc.) Privilege Management

3 Launch the “Anaconda3-2019.10-Linux-x86_64.sh” to install Anaconda

Anaconda3-2019.10-Linux-x86_64.sh
Anaconda3-2019.10-Linux-x86_64.sh

3.1 Accept the license terms and install to user home directory

Accept the license terms and install to user home directory
Accept the license terms and install to user home directory

3.2 Let the installer to initialize Anaconda3 by running conda init by typing “yes” then hit Enter key

Type Yes, then hit Enter key
Type Yes, then hit Enter key

4 Now it’s installed if there is no error

(Installation on Windows and macOS is very simple, just download the correct binary file (.exe) form official website, execute, follow the steps to install, the shortcut will be create automatically on desktop and in start menu, all installation packages can be found here: https://www.anaconda.com/distribution/)


How to: Allow/Setup multiple simultaneous remote users on Ubuntu 19.10 quickly with free/open source software (Under 10 Minutes)

There are many choices for remote session/remote desktop, e.g. How to: Install Xrdp (Remote Desktop) on CentOS 8 – (How to: Use Windows to remote control CentOS 8)

In this guide we will use open source X2Go to achieve this

Table of Contents

  1. Install X2Go Server on Ubuntu
  2. Install tasksel & Desktop environment for remote users
  3. Find out X2Go server’s IP address
  4. Download X2Go client
  5. Configure X2Go client and connect to X2Go server

1 Install X2Go server on Ubuntu

1.1 Launch terminal

1.2 Add the X2Go PPA software repository, update

sudo add-apt-repository ppa:x2go/stable
sudo apt update

1.3 Use following command to install X2Go server

sudo apt-get install x2goserver

2 Install tasksel & Desktop environment for remote users

2.1 Use following command to install tasksel

sudo apt install tasksel

(Currently, gnome 3 is not supported by X2Go, so we have to use other alternative desktop environments for remote users)

2.2 Launch tasksel

sudo tasksel
tasksel
tasksel

Use Up & Down Arrow keys to navigate through the list

2.3 Install desktop environment for remote users

Use Space key to mark the desktop environment, Use Tab key to highlight “<Ok>” then Hit Enter key to begin installation process.

“Xubuntu desktop” is suggested, since Xfce desktop uses less system resources, we can also use “Ubuntu MATE desktop” which uses more system resources than Xfce. We can install one or both of them. In this case, we are installing both “Ubuntu MATE desktop” and “Xubuntu desktop” which is Ubuntu with Xfce deskcp environment.

Selected MATE and Xfce
Selected MATE and Xfce

After hitting OK button, the installation process will begin

Installing packages
Installing packages

When it’s done, it will ask which display manager we want to use. Select the one you prefer, here we chose “gdm3” (Hit Enter key to use gdm3 or use arrow keys to select between gdm3 and lightdm)

Configuring lightdm
Configuring lightdm

It will continue to install packages

Installing packages
Installing packages

2.4 Install Desktop Bindings

This will improve usability, X2Go provides Desktop Binding tools for several desktop environments

For MATE desktop

sudo apt install x2gomatebindings

For LXDE

sudo apt all x2golxdebindings

3 Find out X2Go server’s IP address

3.1 We can use following command in terminal to find out the IP address of X2Go server

ip address

4 Download X2Go client

4.1 We can download X2Go client from heir official website

https://wiki.x2go.org/doku.php/download:start

We can download “mswin” which is for Windows then install, if you want to use another Ubuntu to connect just use “sudo apt install x2goclient” to install the X2G0 client on Ubuntu

PyHoca-GUI can be used for Windows and Ubuntu as well

Download X2Go client
Download X2Go client

5 Configure X2Go client and connect to X2Go server

5.1 Launch X2Go if the “Session preferences – New session” window is not showing, we can click on this icon “New session” to bring up the session configuration window in 5.2

X2Go Main Window
X2Go Main Window

5.2 Session preferences – New session

Session preferences - New session
Session preferences – New session

5.3 Populate the necessary information for connecting the X2Go.

Session name: Any name you like for this session, e.g. Work, Lab etc.

Host: IP address of the X2Go server

Login: Username for login to remote server

Session type: Select MATE for “Ubuntu MATE desktop”, Select Xfce for “Xubuntu desktop”

Session preferences - New session, Configured
Session preferences – New session, Configured

Click OK when done.

5.4 Double click on the session we have just created

X2Go with new session configuration
X2Go with new session configuration

5.5 Enter password for login to the remote server

Enter password
Enter password

5.6 Now it’s successfully connected

X2Go client connected to the server
X2Go client connected to the server

After it’s configured, now we can add multiple users to the X2Go server, so that multiple users will be able to login and use their desktop environment simultaneously.


ls command examples, Introduction to ls command in Linux, How to: Use ls in Linux

ls is a command used to list files and folders within a directory or folder

1 ls command

ls
ls
ls

2 Use long listing format

ls -l

-l: Long listing format

ls with long listing format
ls with long listing format

Column 1: drwxr-xr-x:

The first d means it’s directory if it’s – rather than d means it’s file, if it’s not d or – but l means it’s symbolic link.

Following 9 characters represents the file permission. First 3 rwx represents the file owner’s permission, Second 3 rwx represents the group’s file permission, Third 3 rwx represents others’ file permission

Column 2: 26: How many symbolic link points to this file

Column 3: abc: File/Directory owner

Column 4: abc: File/Directory group

Column 5: 4096: File/Directory size in bytes. For directories, it will always show 4096 bytes

Column 6: Mar 2 22:50: Last time the file was modified

Column 7: annaconda3: File/Directory name

For more information on permission/ Privilege, refer to: Unix/Linux (Ubuntu, Debian, Kali Linux etc.) Privilege Management

3 Show file size

ls -hl
 
or
 
ls -lsi

-h: human readable units (1024)

-si: human readable units (1000)

ls - hl
ls – hl
ls -lsi
ls -lsi

4 Sort the results by size

ls -hlS

-S: sort by file size, largest first

ls -hlS
ls -hlS

5 Specify units

ls -l --block-size=G
K = Kilobyte
M = Megabyte
G = Gigabyte
T = Terabyte
P = Petabyte
E = Exabyte
Z = Zettabyte
Y = Yottabyte
ls -l --block-size=G
ls -l –block-size=G

6 Show hidden files

In Linux, files begin with “.” are hidden. we can use -a switch with ls to show them

ls -a
ls -a
ls -a

7 Only list directories

ls -d */
Only list directories
Only list directories

8 Do not list owner

ls -g
Do not list owner
Do not list owner

9 Do not list group names in a long listing

ls -lG
Do not list group names in a long listing
Do not list group names in a long listing

10 Show UID and GID

ls -n
Show UID and GID
Show UID and GID

11 Print without colour

ls --color=never
Print without colour
Print without colour

12 Show index number of each file (inode)

ls -li
Show index number of each file (inode)
Show index number of each file (inode)

13 Append / indicator to directories

ls -p
Append / indicator to directories
Append / indicator to directories

14 Reverse sorting results

ls -r
Reverse sorting results
Reverse sorting results

15 List subdirectories recursively

ls -R
List subdirectories recursively
List subdirectories recursively

16 Sort alphabetically by entry extension

ls -lX
Sort alphabetically by entry extension method 1
Sort alphabetically by entry extension method 1

OR

ls -sort=extension
Sort alphabetically by entry extension method 2
Sort alphabetically by entry extension method 2

17 Sort by modification time (Latest at the top)

ls -lt
Sort by modification time (Latest at the top)
Sort by modification time (Latest at the top)

18 List folders/files for home directory

ls ~
List folders/files for home directory
List folders/files for home directory

19 List folders/files for parent directory

ls ../
List folders/files for parent directory
List folders/files for parent directory

List directory/files for parent of parent directory

ls ../../
List directory/files for parent of parent directory
List directory/files for parent of parent directory

20 Show ls version information

ls -version
Show ls version information
Show ls version information

Help page of ls

Usage: ls [OPTION]... [FILE]...
List information about the FILEs (the current directory by default).
Sort entries alphabetically if none of -cftuvSUX nor --sort is specified.
Mandatory arguments to long options are mandatory for short options too.
  -a, --all                  do not ignore entries starting with .
  -A, --almost-all           do not list implied . and ..
      --author               with -l, print the author of each file
  -b, --escape               print C-style escapes for nongraphic characters
      --block-size=SIZE      with -l, scale sizes by SIZE when printing them;
                               e.g., '--block-size=M'; see SIZE format below
  -B, --ignore-backups       do not list implied entries ending with ~
  -c                         with -lt: sort by, and show, ctime (time of last
                               modification of file status information);
                               with -l: show ctime and sort by name;
                               otherwise: sort by ctime, newest first
  -C                         list entries by columns
      --color[=WHEN]         colorize the output; WHEN can be 'always' (default
                               if omitted), 'auto', or 'never'; more info below
  -d, --directory            list directories themselves, not their contents
  -D, --dired                generate output designed for Emacs' dired mode
  -f                         do not sort, enable -aU, disable -ls --color
  -F, --classify             append indicator (one of */=>@|) to entries
      --file-type            likewise, except do not append '*'
      --format=WORD          across -x, commas -m, horizontal -x, long -l,
                               single-column -1, verbose -l, vertical -C
      --full-time            like -l --time-style=full-iso
  -g                         like -l, but do not list owner
      --group-directories-first
                             group directories before files;
                               can be augmented with a --sort option, but any
                               use of --sort=none (-U) disables grouping
  -G, --no-group             in a long listing, don't print group names
  -h, --human-readable       with -l and -s, print sizes like 1K 234M 2G etc.
      --si                   likewise, but use powers of 1000 not 1024
  -H, --dereference-command-line
                             follow symbolic links listed on the command line
      --dereference-command-line-symlink-to-dir
                             follow each command line symbolic link
                               that points to a directory
      --hide=PATTERN         do not list implied entries matching shell PATTERN
                               (overridden by -a or -A)
      --hyperlink[=WHEN]     hyperlink file names; WHEN can be 'always'
                               (default if omitted), 'auto', or 'never'
      --indicator-style=WORD  append indicator with style WORD to entry names:
                               none (default), slash (-p),
                               file-type (--file-type), classify (-F)
  -i, --inode                print the index number of each file
  -I, --ignore=PATTERN       do not list implied entries matching shell PATTERN
  -k, --kibibytes            default to 1024-byte blocks for disk usage;
                               used only with -s and per directory totals
  -l                         use a long listing format
  -L, --dereference          when showing file information for a symbolic
                               link, show information for the file the link
                               references rather than for the link itself
  -m                         fill width with a comma separated list of entries
  -n, --numeric-uid-gid      like -l, but list numeric user and group IDs
  -N, --literal              print entry names without quoting
  -o                         like -l, but do not list group information
  -p, --indicator-style=slash
                             append / indicator to directories
  -q, --hide-control-chars   print ? instead of nongraphic characters
      --show-control-chars   show nongraphic characters as-is (the default,
                               unless program is 'ls' and output is a terminal)
  -Q, --quote-name           enclose entry names in double quotes
      --quoting-style=WORD   use quoting style WORD for entry names:
                               literal, locale, shell, shell-always,
                               shell-escape, shell-escape-always, c, escape
                               (overrides QUOTING_STYLE environment variable)
  -r, --reverse              reverse order while sorting
  -R, --recursive            list subdirectories recursively
  -s, --size                 print the allocated size of each file, in blocks
  -S                         sort by file size, largest first
      --sort=WORD            sort by WORD instead of name: none (-U), size (-S),
                               time (-t), version (-v), extension (-X)
      --time=WORD            with -l, show time as WORD instead of default
                               modification time: atime or access or use (-u);
                               ctime or status (-c); also use specified time
                               as sort key if --sort=time (newest first)
      --time-style=TIME_STYLE  time/date format with -l; see TIME_STYLE below
  -t                         sort by modification time, newest first
  -T, --tabsize=COLS         assume tab stops at each COLS instead of 8
  -u                         with -lt: sort by, and show, access time;
                               with -l: show access time and sort by name;
                               otherwise: sort by access time, newest first
  -U                         do not sort; list entries in directory order
  -v                         natural sort of (version) numbers within text
  -w, --width=COLS           set output width to COLS.  0 means no limit
  -x                         list entries by lines instead of by columns
  -X                         sort alphabetically by entry extension
  -Z, --context              print any security context of each file
  -1                         list one file per line.  Avoid '\n' with -q or -b
      --help     display this help and exit
      --version  output version information and exit
The SIZE argument is an integer and optional unit (example: 10K is 10*1024).
Units are K,M,G,T,P,E,Z,Y (powers of 1024) or KB,MB,... (powers of 1000).
The TIME_STYLE argument can be full-iso, long-iso, iso, locale, or +FORMAT.
FORMAT is interpreted like in date(1).  If FORMAT is FORMAT1<newline>FORMAT2,
then FORMAT1 applies to non-recent files and FORMAT2 to recent files.
TIME_STYLE prefixed with 'posix-' takes effect only outside the POSIX locale.
Also the TIME_STYLE environment variable sets the default style to use.
Using color to distinguish file types is disabled both by default and
with --color=never.  With --color=auto, ls emits color codes only when
standard output is connected to a terminal.  The LS_COLORS environment
variable can change the settings.  Use the dircolors command to set it.
Exit status:
 0  if OK,
 1  if minor problems (e.g., cannot access subdirectory),
 2  if serious trouble (e.g., cannot access command-line argument).
GNU coreutils online help: <https://www.gnu.org/software/coreutils/>
Full documentation at: <https://www.gnu.org/software/coreutils/ls>
or available locally via: info '(coreutils) ls invocation'

Man page of ls

LS(1)                                    User Commands                                   LS(1)
NAME
       ls - list directory contents
SYNOPSIS
       ls [OPTION]... [FILE]...
DESCRIPTION
       List  information about the FILEs (the current directory by default).  Sort entries al‐
       phabetically if none of -cftuvSUX nor --sort is specified.
       Mandatory arguments to long options are mandatory for short options too.
       -a, --all
              do not ignore entries starting with .
       -A, --almost-all
              do not list implied . and ..
       --author
              with -l, print the author of each file
       -b, --escape
              print C-style escapes for nongraphic characters
       --block-size=SIZE
              with -l, scale sizes by SIZE when printing  them;  e.g.,  '--block-size=M';  see
              SIZE format below
       -B, --ignore-backups
              do not list implied entries ending with ~
       -c     with -lt: sort by, and show, ctime (time of last modification of file status in‐
              formation); with -l: show ctime and sort by name; otherwise: sort by ctime, new‐
              est first
       -C     list entries by columns
       --color[=WHEN]
              colorize  the  output;  WHEN  can  be  'always' (default if omitted), 'auto', or
              'never'; more info below
       -d, --directory
              list directories themselves, not their contents
       -D, --dired
              generate output designed for Emacs' dired mode
       -f     do not sort, enable -aU, disable -ls --color
       -F, --classify
              append indicator (one of */=>@|) to entries
       --file-type
              likewise, except do not append '*'
       --format=WORD
              across -x, commas -m, horizontal -x, long -l, single-column -1, verbose -l, ver‐
              tical -C
       --full-time
              like -l --time-style=full-iso
       -g     like -l, but do not list owner
       --group-directories-first
              group directories before files;
              can  be augmented with a --sort option, but any use of --sort=none (-U) disables
              grouping
       -G, --no-group
              in a long listing, don't print group names
       -h, --human-readable
              with -l and -s, print sizes like 1K 234M 2G etc.
       --si   likewise, but use powers of 1000 not 1024
       -H, --dereference-command-line
              follow symbolic links listed on the command line
       --dereference-command-line-symlink-to-dir
              follow each command line symbolic link
              that points to a directory
       --hide=PATTERN
              do not list implied entries matching shell PATTERN (overridden by -a or -A)
       --hyperlink[=WHEN]
              hyperlink file names; WHEN can be 'always'  (default  if  omitted),  'auto',  or
              'never'
       --indicator-style=WORD
              append  indicator  with  style  WORD to entry names: none (default), slash (-p),
              file-type (--file-type), classify (-F)
       -i, --inode
              print the index number of each file
       -I, --ignore=PATTERN
              do not list implied entries matching shell PATTERN
       -k, --kibibytes
              default to 1024-byte blocks for disk usage; used only with -s and per  directory
              totals
       -l     use a long listing format
       -L, --dereference
              when showing file information for a symbolic link, show information for the file
              the link references rather than for the link itself
       -m     fill width with a comma separated list of entries
       -n, --numeric-uid-gid
              like -l, but list numeric user and group IDs
       -N, --literal
              print entry names without quoting
       -o     like -l, but do not list group information
       -p, --indicator-style=slash
              append / indicator to directories
       -q, --hide-control-chars
              print ? instead of nongraphic characters
       --show-control-chars
              show nongraphic characters as-is (the default, unless program is 'ls' and output
              is a terminal)
       -Q, --quote-name
              enclose entry names in double quotes
       --quoting-style=WORD
              use  quoting  style  WORD for entry names: literal, locale, shell, shell-always,
              shell-escape, shell-escape-always, c, escape (overrides  QUOTING_STYLE  environ‐
              ment variable)
       -r, --reverse
              reverse order while sorting
       -R, --recursive
              list subdirectories recursively
       -s, --size
              print the allocated size of each file, in blocks
       -S     sort by file size, largest first
       --sort=WORD
              sort by WORD instead of name: none (-U), size (-S), time (-t), version (-v), ex‐
              tension (-X)
       --time=WORD
              with -l, show time as WORD instead of default modification time: atime or access
              or  use  (-u);  ctime  or  status  (-c);  also use specified time as sort key if
              --sort=time (newest first)
       --time-style=TIME_STYLE
              time/date format with -l; see TIME_STYLE below
       -t     sort by modification time, newest first
       -T, --tabsize=COLS
              assume tab stops at each COLS instead of 8
       -u     with -lt: sort by, and show, access time; with -l: show access time and sort  by
              name; otherwise: sort by access time, newest first
       -U     do not sort; list entries in directory order
       -v     natural sort of (version) numbers within text
       -w, --width=COLS
              set output width to COLS.  0 means no limit
       -x     list entries by lines instead of by columns
       -X     sort alphabetically by entry extension
       -Z, --context
              print any security context of each file
       -1     list one file per line.  Avoid '\n' with -q or -b
       --help display this help and exit
       --version
              output version information and exit
       The SIZE argument is an integer and optional unit (example: 10K is 10*1024).  Units are
       K,M,G,T,P,E,Z,Y (powers of 1024) or KB,MB,... (powers of 1000).
       The TIME_STYLE argument can be full-iso, long-iso, iso, locale, or +FORMAT.  FORMAT  is
       interpreted  like  in  date(1).  If FORMAT is FORMAT1<newline>FORMAT2, then FORMAT1 ap‐
       plies to non-recent files and  FORMAT2  to  recent  files.   TIME_STYLE  prefixed  with
       'posix-'  takes  effect only outside the POSIX locale.  Also the TIME_STYLE environment
       variable sets the default style to use.
       Using  color  to  distinguish  file  types  is  disabled  both  by  default  and   with
       --color=never.   With  --color=auto,  ls emits color codes only when standard output is
       connected to a terminal.  The LS_COLORS environment variable can change  the  settings.
       Use the dircolors command to set it.
   Exit status:
       0      if OK,
       1      if minor problems (e.g., cannot access subdirectory),
       2      if serious trouble (e.g., cannot access command-line argument).
AUTHOR
       Written by Richard M. Stallman and David MacKenzie.
REPORTING BUGS
       GNU coreutils online help: <https://www.gnu.org/software/coreutils/>
       Report ls translation bugs to <https://translationproject.org/team/>
COPYRIGHT
       Copyright  ©  2018 Free Software Foundation, Inc.  License GPLv3+: GNU GPL version 3 or
       later <https://gnu.org/licenses/gpl.html>.
       This is free software: you are free to change and redistribute it.  There  is  NO  WAR‐
       RANTY, to the extent permitted by law.
SEE ALSO
       Full documentation at: <https://www.gnu.org/software/coreutils/ls>
       or available locally via: info '(coreutils) ls invocation'
GNU coreutils 8.30                        August 2019                                    LS(1)

How to: Connect SSH without password, How to: Connect SSH with key instead of password

On server 1 (The one needs to be logged into)

1 Use following command to generate necessary rsa files (We can proceed with hitting Enter key for all questions)

ssh-keygen -t rsa
ssh-keygen -t rsa
ssh-keygen -t rsa

When it’s finished, we will get two files under ~/.ssh folder

~/.ssh/id_ras: Private key

~/.ssh/id_rsa.pub: Public key (Needs to be uploaded to the other server)

One server 2(The one we will used to connect/login to server 1)

Before upload the public key file to server 2, we need to make sure ~/.ssh folder exists on server 2

# Creating ~/.ssh folder
mkdir ~/.ssh
chmod 755 ~/.ssh

Upload public key from server 1 to server 2 (10.0.0.2)

scp /root/.ssh/id_rsa.pub [email protected]:/root/.ssh/authorized_keys

scp /username/.ssh/id_rsa.pub username@hostIPorName:/username/.ssh/authorized_keys

Simple usage of xargs, tr, head, tail commands

1 xargs

Similar to -exec

Delete .txt files from current folder

find . -name "*.txt" | xargs rm

Count line numbers in all .txt files

find . -name "*.txt" | xargs wc -l

2 tr

Replace

Replace capital with small case

echo "THIS IS A TEST" | tr "A-Z" "a-z"
 
#Result this is a test

ROT13 Encrypt

echo "Test" | tr "a-zA-Z" "n-za-mN-ZA-m"
 
# Result: Grfg

ROT13 Decrypt

echo "Grfg" | tr "a-zA-Z" "n-za-mN-ZA-m"
 
# Result: Test

Remove digits from the string

echo "This 3 is 1831 a tes2t" | tr -d "0-9"
 
# Result: This  is  a test

Remove redundant space

echo "This is  a       test message     " | tr -s ' '
 
# Result: This is a test message

Remove redundant empty lines form text file

cat text.txt | tr -s '\n'

3 head

Print first 10 lines

head my.log -n 10

4 tail

Print last 10 lines

tail my.log -n 10

Different ways to search in Linux/Debian/Ubuntu/Kali Linux/CentOS/RHEL etc.

locate – Locate files

It can find files quickly but it depends on updatedb to update the index, updatedb runs once everyday but we can run it manually.

# Update the index manually if necessary
sudo updatedb
 
# Find/Search for "myfile"
locate myfile

whereis, which – Locate command/executable files

whereis ProgramName
 
which ProgramName

whereis: Will search for executable, source code and documents from default installation folder (Usually, it’s the default folder when installing with root)

Default folders are:

/bin
/sbin
/usr/bin
/usr/lib
/usr/local/ma
etc.

which: Will show results from environment variables, it’s very useful for finding where the actual program/executable located

e.g. Copy source code to current directory without typing complete path

cp 'which myScript.sh'

If which ProgramName returns /usr/bin/which: no ProgramName in (/home/usr/bin:/bin), it means the ProgramName is not located in environment variables, it can’t be executed directly by it’s name

find – Find files with multiple conditions

Search by folder

# Find files with name "myFile" under root directory
find / -name myFile

If executing the above command with non-root user, it will return many errors (Permission denied), we can use following command to ignore all errors but keep normal output

find / -name 2>/dev/null

What is 2>/dev/null ?: Refer to “I/O redirection in Linux” section in Linux – Basics, Useful Terminal commands and Basic File manipulation, (QuickStart)

Search by time

Show log files which were modified within 5 minutes in current working directory

find . -name '*.log' -mmin -5

(To show log files which were modified within 1 day in current working directory)

find . -name '*.log' -mtime -1

Search by Size

e.g. Search all jpg and bmp files

find . \( -name "*.jpg" -o -name "*.bmp" ) | less
 
or
 
find . -regex ".*\(\.jpg\|\.bmp\)$"
 
# Find files over 10G
find . -type f -size +100G

Limit search folder depth

Search for any .txt files within maximum 3 folder depth

find . -maxdepth 3 -name *.txt

Search for files which are not…

Search for files which are not .txt files

find . -not -name *.txt

find, grep Search for file contents

To list name of all .txt files with Mytext in the content

find . -name *.txt -exec grep -l 'Mytext' {} \;
 
or
 
find . -name *.txt | xargs grep -l 'Mytext'
 
or
 
grep -rl 'Mytext'

To view matched previous 2 lines and next 3 lines

grep -A 3 -B 2 'Mytext' a.txt

To view match counts

# Exact match
grep -c 'Mytext' a.txt
 
# Ignore case
grcp -ci 'Mytext' a.txt

Count number of series in FASTA and FASTQ

grep '^>' test.fa
 
grep '^+$' test.fq

^: Start with …

$: End with …

Match/Show unmatched lines

grep -v 'Mytext' a.txt

Regular Expression

By default, grep uses basic regular expression, add -E switch to use extended regular expression, add -P switch to use perl format regular expression

e.g. Remove all blank lines from text file

grep -v '^$' a.txt >a-without-blank-lines.txt

Extended reading

More about “find” command can be found here: How to: Use “find” command in Linux (Debian, Ubuntu, Kali Linux, CentOS, RHEL/RedHat etc.)

Other techniques related to grep can be found here: How to: Search in Linux, How to: Use grep command, How to: Use grep to search

More on whereis, which commands: Linux, Ubuntu etc. How to find where the program is installed