Skip to content

grep

Character Classes and Bracket Expressions

A bracket expression is a list of characters enclosed by [and]. It matches any single character in that list. If the first character of the list is the caret ^, then it matches any character not in the list, and it is unspecified whether it matches an encoding error. For example, the regular expression [0123456789] matches any single digit, whereas [^()] matches any single character that is not an opening or closing parenthesis, and might or might not match an encoding error.

Within a bracket expression, a range expression consists of two characters separated by a hyphen. It matches any single character that sorts between the two characters, inclusive. In the default C locale, the sorting sequence is the native character order; for example, [a-d] is equivalent to [abcd]. In other locales, the sorting sequence is not specified, and [a-d] might be equivalent to [abcd] or to [aBbCcDd], or it might fail to match any character, or the set of characters that it matches might be erratic, or it might be invalid. To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the LC_ALL environment variable to the value C.

Finally, certain named classes of characters are predefined within bracket expressions, as follows. Their interpretation depends on the LC_CTYPE locale; for example, [[:alnum:]] means the character class of numbers and letters in the current locale.

Basic vs Extended Regular Expressions

In basic regular expressions the meta-characters ?, +, {, |, (, and ) lose their special meaning; instead use the backslashed versions \?, \+, \{, \|, \(, and \).

Exit Status

Normally the exit status is 0 if a line is selected, 1 if no lines were selected, and 2 if an error occurred. However, if the -q or --quiet or --silent is used and a line is selected, the exit status is 0 even if an error occurred.

Character Classes

Character Bracket Expressions
[:digit:] Only the digits 0 to 9
[:alnum:] Any alphanumeric character 0 to 9 OR A to Z or a to z.
[:alpha:] Alphabetic characters: [:lower:] and [:upper:]; in the C locale and ASCII character encoding, this is the same as [A-Za-z]
[:blank:] Blank characters: space and tab.
[:alnum:] Alphanumeric characters: [:alpha:] and [:digit:]; in the C locale and ASCII character encoding, this is the same as [0-9A-Za-z]
[:cntrl:] Control characters. In ASCII, these characters have octal codes 000 through 037, and 177 (DEL). In other character sets, these are the equivalent characters, if any.
[:digit:] Digits: 0 1 2 3 4 5 6 7 8 9.
[:graph:] Graphical characters: [:alnum:] and [:punct:].
[:lower:] Lower-case letters; in the C locale and ASCII character encoding, this is a b c d e f g h i j k l m n o p q r s t u v w x y z.
[:print:] Printable characters: [:alnum:], [:punct:], and space.
[:punct:] Punctuation characters; in the C locale and ASCII character encoding, this is ! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ ] ^ _ ` {
[:space:] Space characters: in the C locale, this is tab, newline, vertical tab, form feed, carriage return, and space. See Usage, for more discussion of matching newlines.
[:upper:] Upper-case letters: in the C locale and ASCII character encoding, this is A B C D E F G H I J K L M N O P Q R S T U V W X Y Z.
[:xdigit:] Hexadecimal digits: 0 1 2 3 4 5 6 7 8 9 A B C D E F a b c d e f.

Repetition

Repetition Description
§A Regular expression may be followed by one of several repetition operators
? The preceding item is optional and matched at most once
* The preceding item will be matched zero or more times.
+ The preceding item will be matched one or more times.
{n} The preceding item is matched exactly n times.
{n,} The preceding item is matched n or more times.
{,m} The preceding item is matched at most m times. This is a GNU extension.
{n,m} The preceding item is matched at least n times, but not more than m times.
] Ends the bracket expression if its not the first list item. So, if you want to make the]` character a list item, you must put it first.
[. Represents the open collating symbol.
.] Represents the close collating symbol.
[= Represents the open equivalence class.
=] Represents the close equivalence class.
[: Represents the open character class symbol, and should be followed by a valid character class name.
:] Represents the close character class symbol.
- Represents the range if its not first or last in a list or the ending point of a range. To make the-` a list item, it is best to put it last.
^ Represents the characters not in the list. If you want to make the ^ character a list item, place it anywhere but first

Grep only uppercase letters

grep "^[[:upper:]]"

Search for strings contains A-Za-z

grep "([A-Za-z ]*)" 

Grep wuzi for three times then quit

grep -E "wuzi{3}" 

Grep two strings

grep -E "(GPL|General Public License)" 

Count the number of matches

grep -c "wuzi" 

Get with grep exact x strings matches from output

grep "^[A-Za-z0-9]\{6\}$"

Grep all non-ascii character in file

grep --color='auto' -P -n '[^\x00-\x7F]' 

Get all domains from html

grep -oP '(?<=www\.)\s?[^\/]*'  

Grep tabs

grep $'\t' 

Search for a php command backdoor

grep -RPl --include=*.{php,txt,asp} \
    "(passthru|shell_exec|system|phpinfo|base64_decode|chmod|mkdir|fopen|fclose|readf?ile) *\(" /var/www/

Find all strings that ends with wuzi

grep "wuzi$" <somefile>.txt   
grep ^[A-D] table.txt

Grep AND & OR

grep 'Manager.*Sales\|Developer*Tech'

Search only for the full word

grep -w "of" table.txt
grep -A 3 "of" table.txt
grep -B 3 "of" table.txt

Select Matches at Ending

grep "fish$"

Match from a Set of Characters

grep "c[aeiou]t"

Specify Repetitions in Pattern

grep -E "[aeiou]{3}"

Display all Sub-directories

ls -l ~ | grep "drw"

Search Text in Files

grep -l 'main'

Find Network Hosts

grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}"

Count Number of Empty Lines

grep -c "^$"

Remove empty lines

grep -v -e '^$'

Remove empty lines

grep -v -e '^[[:space:]]*$'

Grep sterr

command 2>&1 >/dev/null \
    |grep 'something'

Extract between 2 words

echo "Here is a string" \
    |grep -o -P '(?<=Here).*(?=string)'
echo "field1 field2 field3 field4" \
    |grep -oP '(?<=field3 )[^ ]*'
echo "field1 field2 field3 field4" \
    |grep -oP '(?<=field2 )\w+'

Filename

grep -Po '"id":.*?[^\\]",'

Grep wstrings that wich is include 22 words or more

grep `ˆ.\{22,\}$`

Search for multiple patterns, all files in current dir

egrep 'apple|banana|orange'                                                       

Same as above thing, case-insensitive

egrep -i 'apple|banana|orange'                                                                                                                          

All lines matching multiple patterns

egrep 'score|nation|liberty|equal'                         

Grep "Foo" or "Goo"

grep '[FG]oo' <somefile>.txt   

Grep Blank Characters.

grep "^[[:blank:]]     

Find blank lines

grep "^$"

Show all matches, and five lines before each match

grep -B5 "test"

Show all matches, and ten lines after each match

grep -A10 "test" 

Five lines before and ten lines after

grep -B5 -A5 "test"                                                                                                                                

Remove empty lines, no quotes is being used here

cat <somefile>.txt|grep \S 

MAC & IP Adresses

Grep mac addr

egrep -io '[0-9a-f]{12}'

Grep IPV6

grep "^\([0-9a-fA-F]\{0,4\}:\)\{1,7\}[0-9a-fA-F]\{0,4\}$"

Grep IPV4

grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}"

Grep networks hsots

grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}" 

Grep valid ips only

egrep '\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)' 

Match IPv4 Addresses but don't grep dots

egrep '\<([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\>'

Match (0-255).(0-255).(0-255). Format

grep '(\<([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\>\.){3}'

Grep (0-255).(0-255).(0-255).(0-255) Format IPv4 Addresses, dots included

egrep '(\<([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\>\.){3}\<([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\>'
grep -vx -f <(sqlite3 Main.db .dump) <(sqlite3 ${DB} .schema) 

Extract email addresses

grep -E -o "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,6}\b" 

Extract valid IP addresses

grep -E -o "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)"

Extract passwords

grep -i "pwd\|passw" file.txt

Extract users

grep -i "user\|invalid\|authentication\|login" 

# Extract various hashes 

Extract md5 hashes ({32}), sha1 ({40}), sha256({64}), sha512({128})

egrep -oE '(^|[^a-fA-F0-9])[a-fA-F0-9]{32}([^a-fA-F0-9]|$)' \
    |egrep -o '[a-fA-F0-9]{32}' 

Extract valid MySQL-Old hashes

grep -e "[0-7][0-9a-f]{7}[0-7][0-9a-f]{7}" *.txt > mysql-old-hashes.txt

Extract blowfish hashes

grep -e "$2a\$\08\$(.){75}" 

Extract Joomla hashes

egrep -o "([0-9a-zA-Z]{32}):(w{16,32})" 

Extract VBulletin hashes

egrep -o "([0-9a-zA-Z]{32}):(S{3,32})"

Extraxt phpBB3-MD5

egrep -o '$H$S{31}'

Extract Wordpress-MD5

egrep -o '$P$S{31}' 

Extract Drupal 7

egrep -o '$S$S{52}' 

Extract old Unix-md5

egrep -o '$1$w{8}S{22}' 

Extract md5-apr1

egrep -o '$apr1$w{8}S{22}' 

Extract sha512crypt, SHA512(Unix)

egrep -o '$6$w{8}S{86}' *.txt > sha512crypt.txt

Extract e-mails from text files

grep -E -o "\b[a-zA-Z0-9.#?$*_-]+@[a-zA-Z0-9.#?$*_-]+.[a-zA-Z0-9.-]+\b" *.txt 

Extract HTTP URLs from text files

grep http | grep -shoP 'http.*?[" >]' *.txt > http-urls.txt

Match https, ftp, fopher and mailto

grep -E '(((https|ftp|gopher)|mailto)[.:][^ >"  ]*|www.[-a-z0-9.]+)[^ .,;       >">):]' 

if grep returns "Binary file (standard input) matches" use the following approaches

tr '[\000-\011\013-\037177-377]' '.' < *.log |grep -E "Your_Regex" 

Extract Floating point numbers

grep -E -o "^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?$" *.txt > floats.txt

Mail & Passwords

Grep PAM
grep 'pam_unix(sshd:auth): authentication failure;' /var/log/auth.log \
    |awk '{print $14}' \
    |cut -d"=" -f2

Grep emails only

grep -E -o -r "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,6}\b"

Grep emails and passwords

grep -rEiEio "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,6}\b:...*"

HTTP/HTTPS/FTPS/MAIL/SMTP urls étc

1xGrep http/ftp urls

grep -Eoi '<a [^>]+>'

2xGrep http/ftp urls

grep -Eo 'href="[^\"]+"'

3xGrep http/ftp urls

grep -Eo '(http|https)://[^/"]+'

4xGrep http/ftp urls

grep -Eo "(http|https)://[a-zA-Z0-9./?=_-]*"

Grep emails only

grep -srhw "[[:alnum:]]\+@[[:alnum:]]\+"

Credit Cards

Visa

grep -E -o "4[0-9]{3}[ -]?[0-9]{4}[ -]?[0-9]{4}[ -]?[0-9]{4}" 

MasterCard

grep -E -o "5[0-9]{3}[ -]?[0-9]{4}[ -]?[0-9]{4}[ -]?[0-9]{4}" 

American Express

grep -E -o "\b3[47][0-9]{13}\b" *.txt > american-express.txt

Diners Club

grep -E -o "\b3(?:0[0-5]|[68][0-9])[0-9]{11}\b" *.txt > diners.txt

Discover

grep -E -o "6011[ -]?[0-9]{4}[ -]?[0-9]{4}[ -]?[0-9]{4}" 

JCB

grep -E -o "\b(?:2131|1800|35d{3})d{11}\b"

AMEX

grep -E -o "3[47][0-9]{2}[ -]?[0-9]{6}[ -]?[0-9]{5}" 

Extract IDs

Extract Social Security Number (SSN)
grep -E -o "[0-9]{3}[ -]?[0-9]{2}[ -]?[0-9]{4}" 

Extract Indiana Driver License Number

grep -E -o "[0-9]{4}[ -]?[0-9]{2}[ -]?[0-9]{4}" 

Extract US Passport Cards

grep -E -o "C0[0-9]{7}" *.txt > us-pass-card.txt

Extract US Passport Number

grep -E -o "[23][0-9]{8}" *.txt > us-pass-num.txt

Extract US Phone Numberss

grep -Po 'd{3}[s-_]?d{3}[s-_]?d{4}' *.txt > us-phones.txt

Extract ISBN Numbers

egrep -a -o "\bISBN(?:-1[03])?:? (?=[0-9X]{10}$|(?=(?:[0-9]+[- ]){3})[- 0-9X]{13}$|97[89][0-9]{10}$|(?=(?:[0-9]+[- ]){4})[- 0-9]{17}$)(?:97[89][- ]?)?[0-9]{1,5}[- ]?[0-9]+[- ]?[0-9]+[- ]?[0-9X]\b" 
grep -E "(user|username|login|pass|password|pw|credentials)[=:]" /etc/fstab /etc/mtab 2>/dev/null

  • Comments are closed on this article!

Last update: December 6, 2022 16:13:32