How to: Search in Linux, How to: Use grep command, How to: Use grep to search

“grep” is very useful when searching text.

On Microsoft Windows, we can use “dngrep” to achieve similar results

grep command

grep is a command-line utility for searching plain-text data sets for lines that match a regular expression. Its name comes from the ed command g/re/p (globally search a regular expression and print), which has the same effect: doing a global search with the regular expression and printing all matching lines. [1]

Some basic grep usage

grep 'test' filename #Search test in file filename
grep 'test' file1 file2 #Search test from file1 and file2
cat filename | grep 'test' #Print out content from filename then search for test
grep --color=never 'test' filename #Search test from filename and don't highlight the results
grep --color=auto 'test' filename #Search test from filename and displays color in the output unless the output is piped to a command, or redirected to a file
grep --color=always 'test' filename #Search test from filename and highlight the matched string/results
grep 'test' test.txt
grep ‘test’ test.txt
grep --color[auto/never/always] 'test' test.txt
grep –color[auto/never/always] ‘test’ test.txt

Simple search

Search www from /etc/passwd file

grep www /etc/passwd
grep www /etc/passwd
grep www /etc/passwd

Search www from /etc/passwd file with case insensitive switch “-i” (Means WWW www WWw wwW WwW wWw will be included in the results as well)

grep -i 'www' /etc/passwd

Search recursively

Search all files from a folder for a text

e.g. Search “configured” from “/etc” folder

grep -r 'disabled' /etc
 
or
 
grep -R 'disabled' /etc
grep -r 'disabled' /etc
grep -r ‘disabled’ /etc

To eliminate file name

grep -hr 'disabled' /etc
 
or
 
grep -h -r 'disabled' /etc
grep -hr 'disabled' /etc
grep -hr ‘disabled’ /etc

Search with “-w” match only whole words switch

grep -w 'word' filename
grep -w test test.txt
grep -w test test.txt

Use egrep to search two different words

egrep -w 'word1|word2' filename
egrep 'test|testaaa' test.txt
egrep ‘test|testaaa’ test.txt
egrep -w 'test|testaaa' test.txt
egrep -w ‘test|testaaa’ test.txt

Show count of matching results & Show number of the row the results at in the file

#Show count of matching results
grep -c -w test test.txt
#Show number of the row the results at in the file
grep -n -w test test.txt
Show count of matching results, Show number of the row the results at in the file
Show count of matching results, Show number of the row the results at in the file

Search with inverse match (Exclude the string)

grep -v excludeWord filename
grep -v text test.txt
grep -v text test.txt
grep -vn text test.txt
grep -vn text test.txt

Use grep with Linux pipeline

grep -i 'searchTerm' command
command | grep -i 'model'

Show hard drive name

dmesg | egrep '(s|h)d[a-z]'

Show cpu model

#With pipeline
cat /proc/cpuinfo | grep -i 'Model'
 
#Without pipeine
grep -i 'Model' /proc/cpuinfo
grep with/without Linux pipeline
grep with/without Linux pipeline

List files which includes the search term

Search all “.txt” files which contains “test”

grep -l 'test' *.txt
grep -l 'test' *.txt
grep -l ‘test’ *.txt

Some switches of grep command

-cCount of occurrence
-hRemove file name and path from results
-iCase insensitive
-lPrint only names of FILEs with selected lines
-nPrint line numbers
-RRecursive search, obey all symbolic links
-rRecursive search all folders
-vReverse match
-wMatch only whole words
–colorApply/Disable color scheme for search results

Help page of grep

Usage: grep [OPTION]... PATTERNS [FILE]...
Search for PATTERNS in each FILE.
Example: grep -i 'hello world' menu.h main.c
PATTERNS can contain multiple patterns separated by newlines.
Pattern selection and interpretation:
  -E, --extended-regexp     PATTERNS are extended regular expressions
  -F, --fixed-strings       PATTERNS are strings
  -G, --basic-regexp        PATTERNS are basic regular expressions
  -P, --perl-regexp         PATTERNS are Perl regular expressions
  -e, --regexp=PATTERNS     use PATTERNS for matching
  -f, --file=FILE           take PATTERNS from FILE
  -i, --ignore-case         ignore case distinctions in patterns and data
      --no-ignore-case      do not ignore case distinctions (default)
  -w, --word-regexp         match only whole words
  -x, --line-regexp         match only whole lines
  -z, --null-data           a data line ends in 0 byte, not newline
Miscellaneous:
  -s, --no-messages         suppress error messages
  -v, --invert-match        select non-matching lines
  -V, --version             display version information and exit
      --help                display this help text and exit
Output control:
  -m, --max-count=NUM       stop after NUM selected lines
  -b, --byte-offset         print the byte offset with output lines
  -n, --line-number         print line number with output lines
      --line-buffered       flush output on every line
  -H, --with-filename       print file name with output lines
  -h, --no-filename         suppress the file name prefix on output
      --label=LABEL         use LABEL as the standard input file name prefix
  -o, --only-matching       show only nonempty parts of lines that match
  -q, --quiet, --silent     suppress all normal output
      --binary-files=TYPE   assume that binary files are TYPE;
                            TYPE is 'binary', 'text', or 'without-match'
  -a, --text                equivalent to --binary-files=text
  -I                        equivalent to --binary-files=without-match
  -d, --directories=ACTION  how to handle directories;
                            ACTION is 'read', 'recurse', or 'skip'
  -D, --devices=ACTION      how to handle devices, FIFOs and sockets;
                            ACTION is 'read' or 'skip'
  -r, --recursive           like --directories=recurse
  -R, --dereference-recursive  likewise, but follow all symlinks
      --include=GLOB        search only files that match GLOB (a file pattern)
      --exclude=GLOB        skip files that match GLOB
      --exclude-from=FILE   skip files that match any file pattern from FILE
      --exclude-dir=GLOB    skip directories that match GLOB
  -L, --files-without-match  print only names of FILEs with no selected lines
  -l, --files-with-matches  print only names of FILEs with selected lines
  -c, --count               print only a count of selected lines per FILE
  -T, --initial-tab         make tabs line up (if needed)
  -Z, --null                print 0 byte after FILE name
Context control:
  -B, --before-context=NUM  print NUM lines of leading context
  -A, --after-context=NUM   print NUM lines of trailing context
  -C, --context=NUM         print NUM lines of output context
  -NUM                      same as --context=NUM
      --color[=WHEN],
      --colour[=WHEN]       use markers to highlight the matching strings;
                            WHEN is 'always', 'never', or 'auto'
  -U, --binary              do not strip CR characters at EOL (MSDOS/Windows)
When FILE is '-', read standard input.  With no FILE, read '.' if
recursive, '-' otherwise.  With fewer than two FILEs, assume -h.
Exit status is 0 if any line (or file if -L) is selected, 1 otherwise;
if any error occurs and -q is not given, the exit status is 2.
Report bugs to: [email protected]
GNU grep home page: <http://www.gnu.org/software/grep/>
General help using GNU software: <https://www.gnu.org/gethelp/>

man page of grep

GREP(1)                                                                                                     User Commands                                                                                                     GREP(1)
NAME
       grep, egrep, fgrep, rgrep - print lines that match patterns
SYNOPSIS
       grep [OPTION...] PATTERNS [FILE...]
       grep [OPTION...] -e PATTERNS ... [FILE...]
       grep [OPTION...] -f PATTERN_FILE ... [FILE...]
DESCRIPTION
       grep  searches  for  PATTERNS  in  each FILE.  PATTERNS is one or more patterns separated by newline characters, and grep prints each line that matches a pattern.  Typically PATTERNS should be quoted when grep is used in a
       shell command.
       A FILE of “-” stands for standard input.  If no FILE is given, recursive searches examine the working directory, and nonrecursive searches read standard input.
       In addition, the variant programs egrep, fgrep and rgrep are the same as grep -E, grep -F, and grep -r, respectively.  These variants are deprecated, but are provided for backward compatibility.
OPTIONS
   Generic Program Information
       --help Output a usage message and exit.
       -V, --version
              Output the version number of grep and exit.
   Pattern Syntax
       -E, --extended-regexp
              Interpret PATTERNS as extended regular expressions (EREs, see below).
       -F, --fixed-strings
              Interpret PATTERNS as fixed strings, not regular expressions.
       -G, --basic-regexp
              Interpret PATTERNS as basic regular expressions (BREs, see below).  This is the default.
       -P, --perl-regexp
              Interpret PATTERNS as Perl-compatible regular expressions (PCREs).  This option is experimental when combined with the -z (--null-data) option, and grep -P may warn of unimplemented features.
   Matching Control
       -e PATTERNS, --regexp=PATTERNS
              Use PATTERNS as the patterns.  If this option is used multiple times or is combined with the -f (--file) option, search for all patterns given.  This option can be used to protect a pattern beginning with “-”.
       -f FILE, --file=FILE
              Obtain patterns from FILE, one per line.  If this option is used multiple times or is combined with the -e (--regexp) option, search for all patterns given.  The empty file  contains  zero  patterns,  and  therefore
              matches nothing.
       -i, --ignore-case
              Ignore case distinctions in patterns and input data, so that characters that differ only in case match each other.
       --no-ignore-case
              Do  not ignore case distinctions in patterns and input data.  This is the default.  This option is useful for passing to shell scripts that already use -i, to cancel its effects because the two options override each
              other.
       -v, --invert-match
              Invert the sense of matching, to select non-matching lines.
       -w, --word-regexp
              Select only those lines containing matches that form whole words.  The test is that the matching substring must either be at the beginning of the line, or preceded by a non-word constituent character.  Similarly, it
              must be either at the end of the line or followed by a non-word constituent character.  Word-constituent characters are letters, digits, and the underscore.  This option has no effect if -x is also specified.
       -x, --line-regexp
              Select only those matches that exactly match the whole line.  For a regular expression pattern, this is like parenthesizing the pattern and then surrounding it with ^ and $.
       -y     Obsolete synonym for -i.
   General Output Control
       -c, --count
              Suppress normal output; instead print a count of matching lines for each input file.  With the -v, --invert-match option (see below), count non-matching lines.
       --color[=WHEN], --colour[=WHEN]
              Surround the matched (non-empty) strings, matching lines, context lines, file names, line numbers, byte offsets, and separators (for fields and groups of context lines) with escape sequences to display them in color
              on the terminal.  The colors are defined by the environment variable GREP_COLORS.  The deprecated environment variable GREP_COLOR is still supported, but its setting does not have priority.  WHEN is  never,  always,
              or auto.
       -L, --files-without-match
              Suppress normal output; instead print the name of each input file from which no output would normally have been printed.  The scanning will stop on the first match.
       -l, --files-with-matches
              Suppress normal output; instead print the name of each input file from which output would normally have been printed.  The scanning will stop on the first match.
       -m NUM, --max-count=NUM
              Stop  reading  a  file  after  NUM  matching  lines.   If the input is standard input from a regular file, and NUM matching lines are output, grep ensures that the standard input is positioned to just after the last
              matching line before exiting, regardless of the presence of trailing context lines.  This enables a calling process to resume a search.  When grep stops after NUM matching lines,  it  outputs  any  trailing  context
              lines.  When the -c or --count option is also used, grep does not output a count greater than NUM.  When the -v or --invert-match option is also used, grep stops after outputting NUM non-matching lines.
       -o, --only-matching
              Print only the matched (non-empty) parts of a matching line, with each such part on a separate output line.
       -q, --quiet, --silent
              Quiet; do not write anything to standard output.  Exit immediately with zero status if any match is found, even if an error was detected.  Also see the -s or --no-messages option.
       -s, --no-messages
              Suppress error messages about nonexistent or unreadable files.
   Output Line Prefix Control
       -b, --byte-offset
              Print the 0-based byte offset within the input file before each line of output.  If -o (--only-matching) is specified, print the offset of the matching part itself.
       -H, --with-filename
              Print the file name for each match.  This is the default when there is more than one file to search.
       -h, --no-filename
              Suppress the prefixing of file names on output.  This is the default when there is only one file (or only standard input) to search.
       --label=LABEL
              Display  input  actually  coming  from  standard input as input coming from file LABEL.  This can be useful for commands that transform a file's contents before searching, e.g., gzip -cd foo.gz | grep --label=foo -H
              'some pattern'.  See also the -H option.
       -n, --line-number
              Prefix each line of output with the 1-based line number within its input file.
       -T, --initial-tab
              Make sure that the first character of actual line content lies on a tab stop, so that the alignment of tabs looks normal.  This is useful with options that prefix their output to the actual content: -H,-n,  and  -b.
              In order to improve the probability that lines from a single file will all start at the same column, this also causes the line number and byte offset (if present) to be printed in a minimum size field width.
       -u, --unix-byte-offsets
              Report Unix-style byte offsets.  This switch causes grep to report byte offsets as if the file were a Unix-style text file, i.e., with CR characters stripped off.  This will produce results identical to running grep
              on a Unix machine.  This option has no effect unless -b option is also used; it has no effect on platforms other than MS-DOS and MS-Windows.
       -Z, --null
              Output a zero byte (the ASCII NUL character) instead of the character that normally follows a file name.  For example, grep -lZ outputs a zero byte after each file name instead of the  usual  newline.   This  option
              makes  the  output  unambiguous,  even  in  the presence of file names containing unusual characters like newlines.  This option can be used with commands like find -print0, perl -0, sort -z, and xargs -0 to process
              arbitrary file names, even those that contain newline characters.
   Context Line Control
       -A NUM, --after-context=NUM
              Print NUM lines of trailing context after matching lines.  Places a line containing a group separator (--) between contiguous groups of matches.  With the -o or --only-matching option,  this  has  no  effect  and  a
              warning is given.
       -B NUM, --before-context=NUM
              Print  NUM  lines  of  leading  context  before matching lines.  Places a line containing a group separator (--) between contiguous groups of matches.  With the -o or --only-matching option, this has no effect and a
              warning is given.
       -C NUM, -NUM, --context=NUM
              Print NUM lines of output context.  Places a line containing a group separator (--) between contiguous groups of matches.  With the -o or --only-matching option, this has no effect and a warning is given.
   File and Directory Selection
       -a, --text
              Process a binary file as if it were text; this is equivalent to the --binary-files=text option.
       --binary-files=TYPE
              If a file's data or metadata indicate that the file contains binary data, assume that the file is of type TYPE.  Non-text bytes indicate binary data; these are either output bytes that are improperly encoded for the
              current locale, or null input bytes when the -z option is not given.
              By  default, TYPE is binary, and grep suppresses output after null input binary data is discovered, and suppresses output lines that contain improperly encoded data.  When some output is suppressed, grep follows any
              output with a one-line message saying that a binary file matches.
              If TYPE is without-match, when grep discovers null input binary data it assumes that the rest of the file does not match; this is equivalent to the -I option.
              If TYPE is text, grep processes a binary file as if it were text; this is equivalent to the -a option.
              When type is binary, grep may treat non-text bytes as line terminators even without the -z option.  This means choosing binary versus text can affect whether a pattern matches a file.   For  example,  when  type  is
              binary the pattern q$ might match q immediately followed by a null byte, even though this is not matched when type is text.  Conversely, when type is binary the pattern . (period) might not match a null byte.
              Warning:  The -a option might output binary garbage, which can have nasty side effects if the output is a terminal and if the terminal driver interprets some of it as commands.  On the other hand, when reading files
              whose text encodings are unknown, it can be helpful to use -a or to set LC_ALL='C' in the environment, in order to find more matches even if the matches are unsafe for direct display.
       -D ACTION, --devices=ACTION
              If an input file is a device, FIFO or socket, use ACTION to process it.  By default, ACTION is read, which means that devices are read just as if they were ordinary files.  If ACTION is skip,  devices  are  silently
              skipped.
       -d ACTION, --directories=ACTION
              If  an  input  file  is  a  directory,  use ACTION to process it.  By default, ACTION is read, i.e., read directories just as if they were ordinary files.  If ACTION is skip, silently skip directories.  If ACTION is
              recurse, read all files under each directory, recursively, following symbolic links only if they are on the command line.  This is equivalent to the -r option.
       --exclude=GLOB
              Skip any command-line file with a name suffix that matches the pattern GLOB, using wildcard matching; a name suffix is either the whole name, or a trailing part that starts with  a  non-slash  character  immediately
              after  a  slash  (/) in the name.  When searching recursively, skip any subfile whose base name matches GLOB; the base name is the part after the last slash.  A pattern can use *, ?, and [...] as wildcards, and \ to
              quote a wildcard or backslash character literally.
       --exclude-from=FILE
              Skip files whose base name matches any of the file-name globs read from FILE (using wildcard matching as described under --exclude).
       --exclude-dir=GLOB
              Skip any command-line directory with a name suffix that matches the pattern GLOB.  When searching recursively, skip any subdirectory whose base name matches GLOB.  Ignore any redundant trailing slashes in GLOB.
       -I     Process a binary file as if it did not contain matching data; this is equivalent to the --binary-files=without-match option.
       --include=GLOB
              Search only files whose base name matches GLOB (using wildcard matching as described under --exclude).
       -r, --recursive
              Read all files under each directory, recursively, following symbolic links only if they are on the command line.  Note that if no file operand is given, grep searches the working directory.  This  is  equivalent  to
              the -d recurse option.
       -R, --dereference-recursive
              Read all files under each directory, recursively.  Follow all symbolic links, unlike -r.
   Other Options
       --line-buffered
              Use line buffering on output.  This can cause a performance penalty.
       -U, --binary
              Treat  the  file(s)  as binary.  By default, under MS-DOS and MS-Windows, grep guesses whether a file is text or binary as described for the --binary-files option.  If grep decides the file is a text file, it strips
              the CR characters from the original file contents (to make regular expressions with ^ and $ work correctly).  Specifying -U overrules this guesswork, causing all files to be read and passed to the matching mechanism
              verbatim; if the file is a text file with CR/LF pairs at the end of each line, this will cause some regular expressions to fail.  This option has no effect on platforms other than MS-DOS and MS-Windows.
       -z, --null-data
              Treat  input and output data as sequences of lines, each terminated by a zero byte (the ASCII NUL character) instead of a newline.  Like the -Z or --null option, this option can be used with commands like sort -z to
              process arbitrary file names.
REGULAR EXPRESSIONS
       A regular expression is a pattern that describes a set of strings.  Regular expressions are constructed analogously to arithmetic expressions, by using various operators to combine smaller expressions.
       grep understands three different versions of regular expression syntax: “basic” (BRE), “extended” (ERE) and “perl” (PCRE).  In GNU grep there is no difference in available functionality between basic and extended syntaxes.
       In  other  implementations,  basic  regular  expressions  are  less  powerful.  The following description applies to extended regular expressions; differences for basic regular expressions are summarized afterwards.  Perl-
       compatible regular expressions give additional functionality, and are documented in pcresyntax(3) and pcrepattern(3), but work only if PCRE is available in the system.
       The fundamental building blocks are the regular expressions that match a single character.  Most characters, including all letters and digits, are regular expressions that match themselves.  Any meta-character with special
       meaning may be quoted by preceding it with a backslash.
       The period . matches any single character.  It is unspecified whether it matches an encoding error.
   Character Classes and Bracket Expressions
       A  bracket  expression  is  a  list  of characters enclosed by [ and ].  It matches any single character in that list.  If the first character of the list is the caret ^ then it matches any character not in the list; it is
       unspecified whether it matches an encoding error.  For example, the regular expression [0123456789] matches any single digit.
       Within a bracket expression, a range expression consists of two characters separated by a hyphen.  It matches any single character that sorts between the two characters, inclusive, using the locale's collating sequence and
       character  set.   For  example,  in  the  default  C  locale,  [a-d] is equivalent to [abcd].  Many locales sort characters in dictionary order, and in these locales [a-d] is typically not equivalent to [abcd]; it might be
       equivalent to [aBbCcDd], for example.  To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the LC_ALL environment variable to the value C.
       Finally, certain named classes of characters are predefined within bracket expressions, as follows.  Their names are self explanatory,  and  they  are  [:alnum:],  [:alpha:],  [:blank:],  [:cntrl:],  [:digit:],  [:graph:],
       [:lower:],  [:print:],  [:punct:],  [:space:], [:upper:], and [:xdigit:].  For example, [[:alnum:]] means the character class of numbers and letters in the current locale.  In the C locale and ASCII character set encoding,
       this is the same as [0-9A-Za-z].  (Note that the brackets in these class names are part of the symbolic names, and must be included in addition to the brackets delimiting the bracket expression.)  Most meta-characters lose
       their special meaning inside bracket expressions.  To include a literal ] place it first in the list.  Similarly, to include a literal ^ place it anywhere but first.  Finally, to include a literal - place it last.
   Anchoring
       The caret ^ and the dollar sign $ are meta-characters that respectively match the empty string at the beginning and end of a line.
   The Backslash Character and Special Expressions
       The  symbols \< and \> respectively match the empty string at the beginning and end of a word.  The symbol \b matches the empty string at the edge of a word, and \B matches the empty string provided it's not at the edge of
       a word.  The symbol \w is a synonym for [_[:alnum:]] and \W is a synonym for [^_[:alnum:]].
   Repetition
       A regular expression may be followed by one of several repetition operators:
       ?      The preceding item is optional and matched at most once.
       *      The preceding item will be matched zero or more times.
       +      The preceding item will be matched one or more times.
       {n}    The preceding item is matched exactly n times.
       {n,}   The preceding item is matched n or more times.
       {,m}   The preceding item is matched at most m times.  This is a GNU extension.
       {n,m}  The preceding item is matched at least n times, but not more than m times.
   Concatenation
       Two regular expressions may be concatenated; the resulting regular expression matches any string formed by concatenating two substrings that respectively match the concatenated expressions.
   Alternation
       Two regular expressions may be joined by the infix operator |; the resulting regular expression matches any string matching either alternate expression.
   Precedence
       Repetition takes precedence over concatenation, which in turn takes precedence over alternation.  A whole expression may be enclosed in parentheses to override these precedence rules and form a subexpression.
   Back-references and Subexpressions
       The back-reference \n, where n is a single digit, matches the substring previously matched by the nth parenthesized subexpression of the regular expression.
   Basic vs Extended Regular Expressions
       In basic regular expressions the meta-characters ?, +, {, |, (, and ) lose their special meaning; instead use the backslashed versions \?, \+, \{, \|, \(, and \).
EXIT STATUS
       Normally the exit status is 0 if a line is selected, 1 if no lines were selected, and 2 if an error occurred.  However, if the -q or --quiet or --silent is used and a line is selected, the exit status is 0 even if an error
       occurred.
ENVIRONMENT
       The behavior of grep is affected by the following environment variables.
       The  locale  for category LC_foo is specified by examining the three environment variables LC_ALL, LC_foo, LANG, in that order.  The first of these variables that is set specifies the locale.  For example, if LC_ALL is not
       set, but LC_MESSAGES is set to pt_BR, then the Brazilian Portuguese locale is used for the LC_MESSAGES category.  The C locale is used if none of these environment variables are set, if the locale catalog is not installed,
       or if grep was not compiled with national language support (NLS).  The shell command locale -a lists locales that are currently available.
       GREP_OPTIONS
              This variable specifies default options to be placed in front of any explicit options.  As this causes problems when writing portable scripts, this feature will be removed in a future release of grep, and grep warns
              if it is used.  Please use an alias or script instead.
       GREP_COLOR
              This variable specifies the color used to highlight matched (non-empty) text.  It is deprecated in favor of GREP_COLORS, but still supported.  The mt, ms, and mc capabilities of GREP_COLORS have  priority  over  it.
              It  can only specify the color used to highlight the matching non-empty text in any matching line (a selected line when the -v command-line option is omitted, or a context line when -v is specified).  The default is
              01;31, which means a bold red foreground text on the terminal's default background.
       GREP_COLORS
              Specifies the colors and other attributes used to highlight various parts of the output.  Its value is a colon-separated list of capabilities that defaults to  ms=01;31:mc=01;31:sl=:cx=:fn=35:ln=32:bn=32:se=36  with
              the rv and ne boolean capabilities omitted (i.e., false).  Supported capabilities are as follows.
              sl=    SGR  substring for whole selected lines (i.e., matching lines when the -v command-line option is omitted, or non-matching lines when -v is specified).  If however the boolean rv capability and the -v command-
                     line option are both specified, it applies to context matching lines instead.  The default is empty (i.e., the terminal's default color pair).
              cx=    SGR substring for whole context lines (i.e., non-matching lines when the -v command-line option is omitted, or matching lines when -v is specified).  If however the boolean rv capability and the  -v  command-
                     line option are both specified, it applies to selected non-matching lines instead.  The default is empty (i.e., the terminal's default color pair).
              rv     Boolean value that reverses (swaps) the meanings of the sl= and cx= capabilities when the -v command-line option is specified.  The default is false (i.e., the capability is omitted).
              mt=01;31
                     SGR  substring  for  matching  non-empty  text  in  any matching line (i.e., a selected line when the -v command-line option is omitted, or a context line when -v is specified).  Setting this is equivalent to
                     setting both ms= and mc= at once to the same value.  The default is a bold red text foreground over the current line background.
              ms=01;31
                     SGR substring for matching non-empty text in a selected line.  (This is only used when the -v command-line option is omitted.)  The effect of the sl= (or cx= if rv) capability remains active when  this  kicks
                     in.  The default is a bold red text foreground over the current line background.
              mc=01;31
                     SGR  substring for matching non-empty text in a context line.  (This is only used when the -v command-line option is specified.)  The effect of the cx= (or sl= if rv) capability remains active when this kicks
                     in.  The default is a bold red text foreground over the current line background.
              fn=35  SGR substring for file names prefixing any content line.  The default is a magenta text foreground over the terminal's default background.
              ln=32  SGR substring for line numbers prefixing any content line.  The default is a green text foreground over the terminal's default background.
              bn=32  SGR substring for byte offsets prefixing any content line.  The default is a green text foreground over the terminal's default background.
              se=36  SGR substring for separators that are inserted between selected line fields (:), between context line fields, (-), and between groups of adjacent lines when nonzero context is specified (--).  The default  is
                     a cyan text foreground over the terminal's default background.
              ne     Boolean  value  that prevents clearing to the end of line using Erase in Line (EL) to Right (\33[K) each time a colorized item ends.  This is needed on terminals on which EL is not supported.  It is otherwise
                     useful on terminals for which the back_color_erase (bce) boolean terminfo capability does not apply, when the chosen highlight colors do not affect the background, or when EL is too slow or  causes  too  much
                     flicker.  The default is false (i.e., the capability is omitted).
              Note that boolean capabilities have no =... part.  They are omitted (i.e., false) by default and become true when specified.
              See  the  Select Graphic Rendition (SGR) section in the documentation of the text terminal that is used for permitted values and their meaning as character attributes.  These substring values are integers in decimal
              representation and can be concatenated with semicolons.  grep takes care of assembling the result into a complete SGR sequence (\33[...m).  Common values to concatenate include 1 for bold, 4  for  underline,  5  for
              blink,  7  for  inverse,  39  for default foreground color, 30 to 37 for foreground colors, 90 to 97 for 16-color mode foreground colors, 38;5;0 to 38;5;255 for 88-color and 256-color modes foreground colors, 49 for
              default background color, 40 to 47 for background colors, 100 to 107 for 16-color mode background colors, and 48;5;0 to 48;5;255 for 88-color and 256-color modes background colors.
       LC_ALL, LC_COLLATE, LANG
              These variables specify the locale for the LC_COLLATE category, which determines the collating sequence used to interpret range expressions like [a-z].
       LC_ALL, LC_CTYPE, LANG
              These variables specify the locale for the LC_CTYPE category, which determines the type of characters, e.g., which characters are whitespace.  This category also determines the character encoding, that  is,  whether
              text is encoded in UTF-8, ASCII, or some other encoding.  In the C or POSIX locale, all characters are encoded as a single byte and every byte is a valid character.
       LC_ALL, LC_MESSAGES, LANG
              These variables specify the locale for the LC_MESSAGES category, which determines the language that grep uses for messages.  The default C locale uses American English messages.
       POSIXLY_CORRECT
              If  set, grep behaves as POSIX requires; otherwise, grep behaves more like other GNU programs.  POSIX requires that options that follow file names must be treated as file names; by default, such options are permuted
              to the front of the operand list and are treated as options.  Also, POSIX requires that unrecognized options be diagnosed as “illegal”, but since they are not really against the law the default is to  diagnose  them
              as “invalid”.  POSIXLY_CORRECT also disables _N_GNU_nonoption_argv_flags_, described below.
       _N_GNU_nonoption_argv_flags_
              (Here  N  is  grep's numeric process ID.)  If the ith character of this environment variable's value is 1, do not consider the ith operand of grep to be an option, even if it appears to be one.  A shell can put this
              variable in the environment for each command it runs, specifying which operands are the results of file name wildcard expansion and therefore should not be treated as options.  This behavior is available  only  with
              the GNU C library, and only when POSIXLY_CORRECT is not set.
NOTES
       This man page is maintained only fitfully; the full documentation is often more up-to-date.
COPYRIGHT
       Copyright 1998-2000, 2002, 2005-2020 Free Software Foundation, Inc.
       This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
BUGS
   Reporting Bugs
       Email  bug  reports to the bug-reporting address ⟨[email protected]⟩.  An email archive ⟨https://lists.gnu.org/mailman/listinfo/bug-grep⟩ and a bug tracker ⟨https://debbugs.gnu.org/cgi/pkgreport.cgi?package=grep⟩ are avail‐
       able.
   Known Bugs
       Large repetition counts in the {n,m} construct may cause grep to use lots of memory.  In addition, certain other obscure regular expressions require exponential time and space, and may cause grep to run out of memory.
       Back-references are very slow, and may require exponential time.
EXAMPLE
       The following example outputs the location and contents of any line containing “f” and ending in “.c”, within all files in the current directory whose names contain “g” and end in “.h”.  The -n option outputs line numbers,
       the -- argument treats expansions of “*g*.h” starting with “-” as file names not options, and the empty file /dev/null causes file names to be output even if only one file name happens to be of the form “*g*.h”.
         $ grep -n -- 'f.*\.c$' *g*.h /dev/null
         argmatch.h:1:/* definitions and prototypes for argmatch.c
       The only line that matches is line 1 of argmatch.h.  Note that the regular expression syntax used in the pattern differs from the globbing syntax that the shell uses to match file names.
SEE ALSO
   Regular Manual Pages
       awk(1), cmp(1), diff(1), find(1), perl(1), sed(1), sort(1), xargs(1), read(2), pcre(3), pcresyntax(3), pcrepattern(3), terminfo(5), glob(7), regex(7).
   Full Documentation
       A complete manual ⟨https://www.gnu.org/software/grep/manual/⟩ is available.  If the info and grep programs are properly installed at your site, the command
              info grep
       should give you access to the complete manual.
GNU grep 3.4                                                                                                  2019-12-29                                                                                                      GREP(1)

Resources

Wikipedia – grep


How to: Use shortcut keys/Key combinations in Linux Terminal

1 Tab

When entering command, enter beginning of the command, file name or folder name or command option then press “Tab” key, it will complete the rest for you automatically or show all possible results.

2 Ctrl + C

Terminate/Kill the command or process, it will terminate the running process immediately. (signal SIGINT). It can be intercepted by a program, thus the program can clean itself up before exiting or not exit at all.

3 Ctrl + Z

Suspending a process by sending the SIGSTOP signal, it cannot be intercepted by the program.

4 Ctrl + D

Exit the current terminal. If you are using SSH, it will close it. If you are using a terminal directly, it will close the terminal window.

5 Ctrl + L

Clear terminal screen, same effect as “clear” command

6 Ctrl + A

Move the type cursor to the beginning of the line (Same as pressing “Home” key on keyboard)

7 Ctrl + E

Move the type cursor to the end of the line (Same as pressing “End” key on keyboard)

8 Ctrl + U

Wipe the line and move the type cursor to the beginning of the line (Instead of use “Backspace” key to clear the line slowly)

9 Ctrl + K

Wipe the content from the type cursor to the end of the line

10 Ctrl + W

Clear a word

Before Ctrl + W

Before Ctrl + W
Before Ctrl + W

After Ctrl + W

After Ctrl + W
After Ctrl + W

11 Ctrl + Y

It will paste text removed by Ctrl + U, Ctrl + U and Ctrl + K. If you have deleted text by mistake, this will be helpful.

12 Ctrl + P

Review last command, use repetitively to go back further. Many Terminal provides this review function by PageUp key as well. Some provide the review function by using up arrow key as well (↑).

13 Ctrl + N

Similar usage as Ctrl + P but opposite direction, this command navigate to more recent commands. Many Terminal provides this review function by PageDown key as well. Some provide the review function by using down arrow key as well (↓).

14 Ctrl + R

Used for search history commands

Bonus:

Alternatively, we can use “history” command to show all history command

To search from history command, we can use “history | grep searchTerm”


Basics about Network configuration in Linux, IP commands, configuration files etc.

Table of Contents

1 Some useful basic IP commands

1.1 Use network/Interface configuration files to make permanent changes.

For CentOS/RHEL/Fedora etc.

File: /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE="eth0"
BOOTPROTO=static
ONBOOT=yes TYPE="Ethernet"
IPADDR=10.0.0.10
NAME="System eth0"
HWADDR=00:53:78:2C:7D:9E
GATEWAY=10.0.0.1

For Debian/Ubuntu/Kali Linux etc.

File: /etc/network/interfaces

auto eth0
iface eth0 inet static
address 10.0.0.10
netmask 255.255.255.0
gateway 10.0.0.1

Restart network services to make the changes take effect

sudo /etc/init.d/networking restart
 
OR
 
sudo service restart networking
 
OR
 
systemctl restart networking

1.2 Assign IP address to a specific interface (eth0 in this example) (nonpersistent, will be lost after system reboot)

sudo ip addr add 10.0.0.10 dev eth0

1.3 Remove IP address from a specific interface

sudo ip addr del 10.0.0.10/24 dev eth0

1.4 Check IP address

sudo ip addr
 
OR
 
sudo ip addr show
 
OR
 
sudo ifconfig

1.5 Enable Network interface

sudo ip link set eth0 up

1.6 Disable Network interface

sudo ip link set eth0 down

1.7 Check routing table

sudo ip route show

1.8 Add Static route

sudo ip route del 10.0.0.0/24

1.9 Add persistent static routes

For CentOS/RHEL/Fedora etc.

File: /etc/sysconfig/network-scripts/route-eth0

Add following

10.0.0.0/24 via 192.168.5.20 dev eth0

For Debian/Ubuntu/Kali Linux etc.

File: /etc/network/interfaces

Add following

up ip route add 10.0.0.0/24 via 192.168.5.20 dev eth0

Restart network services to make the changes take effect

sudo /etc/init.d/networking restart
 
OR
 
sudo service restart networking
 
OR
 
systemctl restart networking

1.10 Add default gateway

sudo ip route add default via 10.0.0.1

2 Network configuration file

For CentOS/RHEL/Fedora etc.

File: /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
   #Alias name for the NIC
BOOTPROTO={static|dhcp|none|bootp}
   #Boot protocol, static|none;dhcp
IPADDR=192.168.10.10
   #Set IP address
NETMASK=255.255.255.0
   #Netmask
GATEWAY=192.168.10.1
   #Gateway
ONBOOT=yes|no
   #Activate the network port or not, on boot
HWADDR=00:1E:0B:8F:B0:D0
   #MAC address, if same as the default MAC address of the hardware, this line can be omitted 
DNS1=202.106.0.20
   #Specifiy DNS server
USERCTL=yes|no
   #Users (non-admin/root) allowed to enable/disable this port or not
PEERDNS=yes|no
   #Accept/Reject the DNS server from DHCP while BOOTPROTO is dhcp

For Debian/Ubuntu/Kali Linux etc.

File: /etc/network/interfaces

auto eth1
     #Automatically connect to Ethernet on boot
iface eth1 inet static
     #Assign IP address by static/dhcp
address 192.168.72.8
     #IP address
netmask 255.255.255.0
     #Netmask
gateway 192.168.72.1
     #Default gateway
dns-nameservers 8.8.8.8 4.4.2.2
     #DNS server

3 Hosts configuration

File: /etc/hosts

192.168.0.10 internalserver.mynet

4 Network Interface Controller (NIC) Naming

lo: Localhost loop

ppp#: Point-to-Point Protocol

eth: Ethernet

5 Network management tool

network tool and NetworkManager tool.

network

Restart network

sudo /etc/init.d/network restart

NetworkManager

It can be used to manager network easily, when X Window is not available this GUI tool can be used to manage network without the need to edit configuration file manually.

nmtui
nmtui
nmtui
NetworkManager TUI - nmtui
NetworkManager TUI – mntui

6 NetworkManager cli

NetworkManager provides cli tools as well alongside nmtui

nmcli con show
     #Get UUID table
nmcli dev
     #Check network device status
nmcli r wifi off
     #Turn off wifi
nmcli - commands
nmcli – commands

Start NetworkManager on boot

chkconfig NetworkManager on
 
OR
 
 systemctl enable NetworkManager

Start NetworkManager immediately

service NetworkManager start
 
OR
 
 systemctl start NetworkManager

How to: Find which process is causing high CPU usage Linux/Ubutu/Debian/Kali Linux/CentOS/RHEL

“top” command

top shows CPU usage in real time.

By default, it lists process by their CPU usage, refreshes every 5 seconds.

We can use following command to show top 10 processes with highest CPU usage.

top -b | head -10
[email protected]:~# top -b | head -10
top - 02:05:40 up 20:24,  1 user,  load average: 0.00, 0.00, 0.00
Tasks: 165 total,   1 running, 164 sleeping,   0 stopped,   0 zombie
%Cpu(s):  3.1 us,  3.1 sy,  0.0 ni, 93.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   3913.3 total,   2066.4 free,    617.4 used,   1229.6 buff/cache
MiB Swap:   4094.0 total,   3850.2 free,    243.8 used.   3042.1 avail Mem 
    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
  61649 root      20   0    9144   3560   3164 R   6.7   0.1   0:00.01 top
      1 root      20   0  166420   5908   3936 S   0.0   0.1   0:04.54 systemd
      2 root      20   0       0      0      0 S   0.0   0.0   0:00.03 kthreadd
  • -b : Batch mode.
  • head -10: Display first 10 lines in the output.
  • PID : Unique ID of the process.
  • USER : Owner of the process.
  • PR : priority of the process.
  • NI : The NICE value of the process.
  • VIRT : How much virtual memory used by the process.
  • RES : How much physical memory used by the process.
  • SHR : How much shared memory used by the process.
  • S : This indicates the status of the process: S=sleep R=running Z=zombie.
  • %CPU : The percentage of CPU used by the process.
  • %MEM : The percentage of RAM used by the process.
  • TIME+ : How long the process being running.
  • COMMAND : Name of the process.

“ps” command

“ps” stands for “processes status”, it display the information about the active/running processes on the system.

We can use following command to find out high CPU usage processes.

ps -eo pid,ppid,%mem,%cpu,cmd --sort=-%cpu | head
[email protected]:~# ps -eo pid,ppid,%mem,%cpu,cmd --sort=-%cpu | head
    PID    PPID %MEM %CPU CMD
     78       2  0.0  0.4 [kswapd0]
    639     576  4.1  0.4 /usr/lib/xorg/Xorg :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch
    262       2  0.0  0.2 [kworker/1:1H-kblockd]
    267       2  0.0  0.1 [kworker/0:1H-kblockd]
    889     823  0.8  0.1 xfwm4 --display :0.0 --sm-client-id 24f144caf-a490-40f0-afc0-fd75665210e1
      1       0  0.1  0.0 /sbin/init splash
      2       0  0.0  0.0 [kthreadd]
      3       2  0.0  0.0 [rcu_gp]
      4       2  0.0  0.0 [rcu_par_gp]

To see the command name instead of full path.

ps -eo pid,ppid,%mem,%cpu,comm --sort=-%cpu | head
[email protected]:~# ps -eo pid,ppid,%mem,%cpu,comm --sort=-%cpu | head
    PID    PPID %MEM %CPU COMMAND
     78       2  0.0  0.4 kswapd0
    639     576  4.1  0.4 Xorg
    262       2  0.0  0.2 kworker/1:1H-kblockd
    267       2  0.0  0.1 kworker/0:1H-kblockd
    889     823  0.8  0.1 xfwm4
      1       0  0.1  0.0 systemd
      2       0  0.0  0.0 kthreadd
      3       2  0.0  0.0 rcu_gp
      4       2  0.0  0.0 rcu_par_gp
  • -e : Select all processes.
  • -o : To customize a output format.
  • –sort=-%cpu : Sort the ouput based on CPU usage.
  • head : To display first 10 lines of the output
  • PID : Unique ID of the process.
  • PPID : Unique ID of the parent process.
  • %MEM : The percentage of RAM used by the process.
  • %CPU : The percentage of CPU used by the process.
  • Command : Name of the process.

htop

htop is a command line utility that allows you to interactively monitor your system’s vital resources or server’s processes in real time

You might need to install htop first

#Debian/Ubuntu/Kali Linux etc.
sudo apt install htop
 
#CentOS/RHEL etc.
sudo yum install htop

To launch htop (Use “q” key to exit)

htop
htop
htop

We can easily sort the processes by their Priority, Nice, Virtual memory usage, Memory, CPU and running time etc. Simply by clicking on the column header.

glances

glances is another utility which can be easily installed and used system resource monitoring tool.

Install

#Debian/Ubuntu/Kali Linux etc.
sudo apt install glances
 
#CentOS/RHEL etc.
sudo yum install glances 

Launch (Use “q” key to exit)

glances

How to: compare Files/Code/Folders/Images/Pictures – List of three Open source Comparing Software/Tools

WinMerge

WinMerge
WinMerge
  • Windows
  • Open source/Free
  • Compare text files/source code files/folders/images
  • Easy to use

WinMerge

Notepad ++

Notepad++ with Compare plugin
Notepad++ with Compare plugin
  • Windows
  • Open source/Free
  • Compare text files/source code files
  • Easy to use

Notepad++

By default Notepad++ doesn’t have compare function.

We can make it possible by easily installing a compare plugin after Notepad++ is installed.

“Plugins -> Plugins Admin…-> Search for “Compare -> Check “Compare” -> Click “Install” button. -> Click on Yes when the pop-up window appears” After it’s done, Notepad++ will restart itself.

Notepad++ -> Plugins Admin
Notepad++ -> Plugins Admin
Notepad++ Install Compare plugin
Notepad++ Install Compare plugin
Click on Yes
Click on Yes
Wait until it's downloaded and installed
Wait until it’s downloaded and installed

Now we can see the Compare plugin is installed. “Plugins -> Compare”

Compare plugin installed
Compare plugin installed

Diffuse

Diffuse
Diffuse
  • Windows/Linux
  • Open source/Free
  • Compare text files/source code files
  • Easy to use

Diffuse

DiffImg

DiffImg
DiffImg
  • Windows/Linux
  • Open source/Free
  • Compare Image files
  • Easy to use

DiffImg

There are many free online text/source code compare and image compare tools as well which doesn’t require any installation, can be handy when it needs to be done quickly, not suitable for huge files or images or any confidential content.


How to: Use “find” command in Linux (Debian, Ubuntu, Kali Linux, CentOS, RHEL/RedHat etc.)

Table of Contents

  • 1 Find command basics
  • 2 Find files and folders by their size
  • 3 Find files by their Owner/Group
  • 4 Find files and folders by date and time
  • 5 Find files by their privileges

1 Find command basics

1.1 Find file from current folder

(“Permission denied” error will appear if the current user doesn’t have permission to access that folder)

Find file named “1.txt” within current working folder

find filename
find 1.txt
find 1.txt
find 1.txt

1.2 Find the file from a folder

Find “1.txt” file under “/tmp” folder

find /path/to/folder -name filename
find /tmp -name 1.txt
find file from a parent folder
find file from a parent folder

1.3 Find file (Ignoring the case/Case insensitive)

Find “1.txt” within “/tmp” folder, ignoring case.

find /path/to/folder -iname filename
find /tmp -iname 1.txt
Find file (Ignoring the case/Case insencitive)
Find file (Ignoring the case/Case insencitive)

1.4 Find folder/directory only

find /path/to/folder -type d -name foldername
find /tmp -type d -name abc
Find folder/directory
Find folder/directory only
Find folder/directory, ignoring the case
Find folder/directory only, ignoring the case

1.5 Find file only

find /path/to/folder -type f -name 1
find /tmp -type f -name 1.txt
Find file only
Find file only

1.6 Find specific file

Find .txt files from “/tmp” folder

find -type f -name "*.extension"
find -type f -name "*.txt"
Find specific file
Find specific file

2 Find files and folders by their size

2.1 Find all 100MB files

find / -size 100M

2.2 Find files which is between 10MB and 100MB

find / -size +50M -size -100M

2.3 Find and delete files which is greater than 500MB in size

find / -size +500M -exec rm -rf {} \;

2.4 Find specific files and delete them

Find all mp3 files which are greater than 5MB, delete them

find / -type f -name *.mp3 -size +5M -exec rm {} \;

3 Find files by their Owner/Group

3.1 Find files by their owner

Find all of the “1.txt” file which belongs to Bob

find / -user Bob -name 1.txt

3.2 Find all files belongs to the user

Find all files belongs to Bob under “/test” folder

find /test -user Bob

3.3 Find files based on their group

Find all files under “/test” which belongs to the group “testgroup”

find /test -group testgroup

3.4 Find all “.txt” files under “/test” which belongs to Bob

find /test -user Bob -iname "*.txt"

4 Find files and folders by date and time

4.1 Find files

4.1 Find files which their data was last modified n*24 hours ago. (3 days in this example)

find / -mtime 3

4.2 Find files being accessed in 3 days

find / -atime 3

4.3 Find files that being modified between last 3-10 days

find / -mtime +3 -time -10

4.4 Find files which their status was last changed 5 minutes ago

find / -cmin -5

4.5 Find files which their data was last modified 5 minutes ago

find / -mmin -5

4.6 Find files which are being access within 5 minutes

find / -amin -5

5 Find files by their privileges

5.1 Find files with “777” permission

find -type f -perm 0777 -print

5.2 Find files without “777” permission

find / -type f ! -perm 777

5.3 Find files with “644” permission (SGID)

find / -perm 2644

5.4 Find files with “551” Sticky Bit permission

find / -perm 1551

5.5 Find SUID files

find / -perm /u=s

5.6 Find read only permission files

find / -perm u=r

5.7 Find executable files

find / -perm /a=x

5.8 Find all files with “777” permission and chmod to 644

find /test -type f -perm 0777 -print -exec chmod 644 {} \;

5.9 Find all folders with “777” permission and chmod to 755

find / -type d -perm 777 -print -exec chmod 755 {} \;

5.10 Find and delete single file (1.txt in this case)

find -type f -name "1.txt" -exec rm -f {} \;

5.11 Find and delete multiple files with same extension (txt files in this case)

find -type f -name "*.txt" -exec rm -f {} \;

5.12 Find all empty files

find / -type f -empty

5.13 Find all empty folders

find / -type d -empty

5.14 Find all hidden files under “/tmp” folder

find /tmp -type f -name ".*"

Quick Linux File Manipulation Commands Reference

  • cd: Change directory
    • cd ~ or cd : Back to top folder
    • cd .. : Uppler level folder (../.. Uppler uppler)
  • pwd : Print working directory (Show current working directory)
  • ls : List files/folders within current folder
  • ls -l : With Detail
  • ls -a : Show all files, including hidden files (starts with dot “.”)
  • ls -lh : Show human readable size units
  • mkdir : Make directory
    • mkdir test : Create a folder named “test”
  • touch : Create file
    • touch mytext.txt : Create a text file named “mytext.file”
  • rm : Remove file, folder
    • rm -r : Remove folder and everything within that folder.
    • rm -f : Remove without confirming
    • rm -i : Confirm before removing
    • rm -rf : Remove the folder and everything within the folder without confirmation
    • rm -rf ./* : Remove everything within the current folder
  • mv : Move, Rename
    • mv file /folder1 : Move “file” under current folder to “folder1” withing current folder
    • mv filename1 filename2 : Rename from “filename1” to “filename2”
  • cp : Copy
    • cp /tmp/file ./ : Copy file under /tmp folder to current folder
    • cp -r myfolder/ folder2 : Copy everything from “myfolder” to “folder2”
  • find : Find/Search
    • find /tmp -name keyword : Search for “keyword” via name within “/tmp folder”
    • find /tmp -name ‘keyword*’ : Search for file names’ begin with “keyword” within “/tmp folder”
    • find /tmp -name ‘?keyword’ : Search for file names’ end with “keyword” within “/tmp folder”
    • find /tmp -size +3M : Search for files which is bigger than 3M
  • which : Show command path
    • which squid
  • cat : Show file content
    • cat /tmp/file : Show content within “/tmp/file”
  • more : Show content in multiple pages
    • more /etc/services
    • “Enter” key to show one more line
    • “Space” key to show one more page
    • “Q” key to exit
  • head : Show the head (top) of the file
    • head -n 10 /etc/services : Show first 10 lines of the “/etc/services” file
  • tail : Show the end of the file
    • tail -n 10 /etc/services : Show last 10 lines of “/etc/services” file
  • ln : Create link
    • ln -s SourceFile NewFile : Create soft link
    • ln SourceFile NewFile : Create hard link
''' Soft link/Symbolic link '''
# When the target file is removed, the soft link is useless
# Soft links can span file systems
touch
''' Soft link/Symbolic link '''
# When the target/source file is removed, the soft link is useless
# Soft links can span file systems
touch 1.txt
echo 'Some text' >> 1.txt #Write 'Some text' to 1.txt file
cat 1.txt #Output content from 1.txt
(Output) 123
ln -s 1.txt 2.txt
ls -l
(Output) 2.txt -> 1.txt #Editing 2.txt will actually edit 1.txt file, size of 2.txt won't change, only the target file 1.txt will change the size.
(Output) 1.txt
''' Hard link '''
# Delete any target/source file, the other one will still exist
# Hard links can't span file systems
touch 1.txt
echo 'Some text' >> 1.txt
ln 1.txt 2.txt
ls -ls
(Output) 1.txt
(Output) 2.txt
echo 'test 2' >> 1.txt
echo 'test 3' >> 2.txt
cat 1.txt
(Output) Some text
(Output) test 2
(Output) test 3
cat 2.txt
(Output) Some text
(Output) test 2
(Output) test 3
# Change the target/source or the hard link file will change the other one as well. Same size, with different name.

Quick Linux diagnostic commands (System Resource & Network)

System Resources

CPU:  uptime, top
RAM:  free, top
Disk: iostat, df -h, ds -S /home | sort -rn head -n 10

Network

NIC info:                   ethtool eth0
Interface info:             ifconfig eth0
routing info:               route -n
Internet Connection:        ping bing.com -c 5
DNS check 1:                dig bing.com
DNS check 2:                host bing.com
DNS check 3:                nslookup bing.com
WAN connection 1:           ping bing.com
WAN connection 2:           traceroute bing.com
Port Open/Close:            nmap -p 80 bing.com
Check host listening on:    netstat -tunlp | grep -w 80

Config Network

DNS:                            /etc/resolv.conf
Add default route:              route add default gw 10.0.0.1
Add route 192.168.0.0/24:       route add -net 192.168.0.0 netmask 255.255.0.0 dev eth0
Remove route 192.168.0.0/23:    route del -net 192.168.0.0 netmask 255.255.0.0 dev eth0
Restart network service 1:      /etc/init.d/networking restart
Restart network service 2:      service restart networking
Restart network service 3:      systemctl restart networking
Bring up NIC 1:                 ifup eth0
Bring up NIC 2:                 ip link set eth0 up
Stop NIC 1:                     ifdown eth0
Stop NIC 2:                     ip link set eth0 down
Host name 1:                    /etc/sysconfig/network
Host name 2:                    /etc/hosts

Useful commands to look up system information, install package etc. (Linux, Debian, Ubuntu, Kali Linux, RedHat, CentOS etc.)

System

uname -a               #Show kernel Info/Operating System/CPU Info
head -n 1 /etc/issue   #Show operating system Name
cat /proc/cpuinfo      #Show detail about CPU
hostname               #Show host name
lspci -tv              #List all PCI devices
lsusb -tv              #List all USB devices
lsmod                  #List all loaded kernel modules
env                    #Show environment variables

Processes

top        #Show real-time status of processes
ps -ef     #Show all processes

Services

systemctl list-units                        #List all running services
 systemctl list-units -a                        #List all services, including inactive units
 systemctl list-units -a --state=inactive    #List all inactive units

Users

w                       #Show active users
id <UserName>           #Show info about the user
last                    #Show user login log
cut -d: -f1 /etc/passwd #Show all users
cut -d: -f1 /etc/group  #Show all groups
crontab -l              #Show all cron jobs (Scheduled jobs) for current user

Resources

free -m                       #Show RAM, SWAP usage
df -h                         #Show partition usage
du -sh                        #Show folder/directory size
grep MemTotal /proc/meminfo   #Show total RAM
grep MemFree /proc/meminfo    #Show free/available RAM
uptime                        #Show system uptime, users, load
cat /proc/loadavg             #Show system load

Disk & Partition

mount | column -t    #Show mounted partition info
fdisk -l             #Show all partitions
swapon -s            #Show all swap partition info

Networking

ifconfig        #Show all interface properties
iptables -L     #Show firewall (iptable) configuration
route -n        #Show routing table
netstat -lntp   #Show all listening ports
netstat -antp   #Show all established connections
netstat -s      #Show statistics about Ethernet

apt-get

update            #Update package list
upgrade           #Upgrade packages
install           #Install new packages
remove            #Remove packages
autoremove        #Remove packages that were automatically installed to satisfy dependencies for other packages and are now no longer needed
purge             #Remove package and configuration files
source            #fetch source packages
build-dep         #Install all dependencies for 'packagename'
dist-upgrade      #Upgrade distro
full-upgrade      #Upgrade distro (For newer 'apt')
dselect-upgrade   #Resolves the delta between the currently-installed packages and the states requested in the list of available packages, and performs the necessary actions to reconcile the two.
clean             #Clears out the local repository of retrieved package files
autoclean         #Another method used to clear out the local repository of downloaded package files, just like clean . The difference between clean and autoclean is that the latter only removes package files that can no longer be downloaded from their sources, and are very likely to be useless.
check             #It does an update of the package lists and checks for broken dependencies
-h        #Local help document
-q        #Output to log (Do not indicate progress)
-qq       #Only output errors
-d        #Only download (Do not Extract or Install)
-s        #Simulate the command, does not install the package for real
-y        #Answer Yes for all commands
-f        #Try to fix dependency issues
-m        #Try to continue even the archive can't be located
-u        #Also show the list of upgrade package
-b        #Compile after getting the source (-V to show detailed version number)
-c=?      #Show the configuration file
-o=?      #Set custom/arbitrary configuration e.g. -d dir::cache=/tmp
#Examples
 
apt-cache search PACKAGE                 #Search package
apt-cache show PACKAGE                   #Get details about the package (Manual, size, version etc.)
apt list --installed                     #List all installed packages
apt-get install PACKAGE                  #Install the packag
apt-get install PACKAGE --reinstall      #Reinstall the package
apt-get -f install                       #Repair install (-f = –fix-missing)
apt-get remove PACKAGE                   #Remove package
apt-get remove PACKAGE --purge           #Remove package and configuration files
apt-get update                           #Update source info (repositories and PPAs)
apt-get upgrade                          #Upgrade installed packages
apt-get dist-upgrade                     #Upgrade system
apt full-upgrade                         #Upgrade system (Newer apt)
apt-get dselect-upgrade                  #Use dselect to upgrade
apt-cache depends PACKAGE                #List the dependencies in human form
apt-cache showpkg PACKAGE                #find the details about dependencies with
apt-cache rdepends PACKAGE               #Searches through the APT cache to locate dependencies for an application, and knows how to emulate the results
apt-get build-dep PACKAGE                #Install all dependencies for 'PACKAGE'
apt-get source PACKAGE                   #Download the source for this PACKAGE
apt-get clean                            #Clears out the local repository of retrieved package files
apt-get autoclean                        #Another method used to clear out the local repository of downloaded package files, just like clean . The difference between clean and autoclean is that the latter only removes package files that can no longer be downloaded from their sources, and are very likely to be useless.
apt-get check                            #It does an update of the package lists and checks for broken dependencies

RPM

rpm -qa                                      #List all installed packages
rpm -qpi Linux-1.4-6.i368.rpm                #Show detail about the package
rpm -qpl Linux-1.4-6.i368.rpm                #Show dependencies
rpm -ivh Linux-1.4-6.i368.rpm                #Begin to inatll
rpm -ivh -replacepkgs Linux-1.4-6.i368.rpm   #Force to install
rpm -e PACKAGE                               #Uninstall package
rpm -uvh filename                            #Upgrade by patch (The filename must be the upgrade patch not package)
rpm -i https://contoso.com/packagename.rpm   #Install package from internet
rpm -Va                                      #Search for corrupted filename
rpm -qf filename                             #Find out which package does the file belongs to