Users Online
· Members Online: 0
· Total Members: 188
· Newest Member: meenachowdary055
Forum Threads
Latest Articles
Articles Hierarchy
Bash Cookbook
Bash Cookbook
# Bash Cookbook
<a href="https://www.packtpub.com/application-development/bash-cookbook?utm_source=github&utm_medium=repository&utm_campaign=9781788629362"><img src="https://d255esdrn735hr.cloudfront.net/sites/default/files/imagecache/ppv4_main_book_cover/B09892.png" alt="Bash Cookbook" height="256px" align="right"></a>
This is the code repository for [Bash Cookbook](https://www.packtpub.com/application-development/bash-cookbook?utm_source=github&utm_medium=repository&utm_campaign=9781788629362), published by Packt.
**Leverage Bash scripting to automate daily tasks and improve productivity**
## What is this book about?
In Linux, one of the most commonly used and most powerful tools is the Bash shell. With its collection of engaging recipes, Bash Cookbook takes you through a series of exercises designed to teach you how to effectively use the Bash shell in order to create and execute your own scripts.
This book covers the following exciting features:
* Understand the basics of Bash shell scripting on a Linux system
* Gain working knowledge of how redirections and pipes interact
* Retrieve and parse input or output of any command
* Automate tasks such as data collection and creating and applying a patch
* Create a script that acts like a program with different features
If you feel this book is for you, get your [copy](https://www.amazon.com/dp/1788629361) today!
<a href="https://www.packtpub.com/?utm_source=github&utm_medium=banner&utm_campaign=GitHubBanner"><img src="https://raw.githubusercontent.com/PacktPublishing/GitHub/master/GitHub.png"
alt="https://www.packtpub.com/" border="5" /></a>
## Instructions and Navigations
All of the code is organized into folders. For example, Chapter02.
The code will look like the following:
```
#!/bin/bash
AGE=17
if [ ${AGE} -lt 18 ]; then
echo "You must be 18 or older to see this movie"
fi
```
**Following is what you need for this book:**
The Bash Cookbook is for you if you are a power user or system administrator involved in writing Bash scripts in order to automate tasks. This book is also ideal if you are interested in learning how to automate complex daily tasks.
With the following software and hardware list you can run all code files present in the book (Chapter 1-8).
### Software and Hardware List
| Chapter | Hardware required | OS required |
| -------- | ------------------------------------| -----------------------------------|
| 1-8 | Atleast 25 GB Free Storage Space, Access to either a DVD or USB flash drive | Linux Ubuntu 16.04 |
### Related products <Other books you may enjoy>
* Mastering Bash [[Packt]](https://www.packtpub.com/networking-and-servers/mastering-bash?utm_source=github&utm_medium=repository&utm_campaign=9781784396879) [[Amazon]](https://www.amazon.com/dp/1784396877)
* Mastering Linux Shell Scripting - Second Edition [[Packt]](https://www.packtpub.com/virtualization-and-cloud/mastering-linux-shell-scripting-second-edition?utm_source=github&utm_medium=repository&utm_campaign=9781788990554) [[Amazon]](https://www.amazon.com/dp/1788990552)
## Get to Know the Authors
**Ron Brash** was the CTO and co-founder of a successful technology consultancy company that provides services in a multitude of domains, but primarily in FOSS and Linux. For over 7 years, he has worked on embedded systems, which provide security and network connectivity in industrial control systems and SCADA networks, all running an optimized embedded Linux. He participates regularly at FOSS and community events, providing feedback and mini-seminars where appropriate. He loves to share knowledge.
**Ganesh Naik** is an author, consultant, and corporate trainer for embedded Android, embedded Linux, IoT, and ML-related product development. He has over 20 years of experience and project accomplishment in IT. He has been a corporate trainer for Indian Space Research Organization, Intel, GE, Samsung, Motorola, Penang Skills Development Center, and various firms in Singapore and India. He has started a company called Levana Technologies, which works with organizations for consulting and training activities.
## Other books by the authors
* [Learning Linux Shell Scripting](https://www.packtpub.com/networking-and-servers/learning-linux-shell-scripting?utm_source=github&utm_medium=repository&utm_campaign=9781785286216)
* [Learning Linux Shell Scripting - Second Edition](https://www.packtpub.com/networking-and-servers/learning-linux-shell-scripting-second-edition?utm_source=github&utm_medium=repository&utm_campaign=9781788993197)
### Suggestions and Feedback
[Click here](https://docs.google.com/forms/d/e/1FAIpQLSdy7dATC6QmEL81FIUuymZ0Wy9vH1jHkvpY57OiMeKGqib_Ow/viewform) if you have any feedback or suggestions.
Chapter 01
01_script.sh
#!/bin/bash
# Pound or hash sign signifies a comment (a line that is not executed)
whoami #Command returning the current username
pwd #Command returning the current working directory on the filesystem
ls # Command returning the results (file listing) of the current working directory
echo “Echo one 1”; echo “Echo two 2” # Notice the semicolon used to delimit multiple commands in the same execution.
-----------------
02_my_first_script.sh
#!/bin/bash
# Echo this is my first comment
echo "Hello world! This is my first Bash script!"
echo -n "I am executing the script with user: "
whoami
echo -n "I am currently running in the directory: "
pwd
exit 0
--------------------
03_example1.sh
#!/bin/bash
PI=3.14
VAR_A=10
VAR_B=$VAR_A
VAR_C=${VAR_B}
echo "Lets print 3 variables:"
echo $VAR_A
echo $VAR_B
echo $VAR_C
echo "We know this will break:"
echo "0. The value of PI is $PIabc" # Breaks - the interpreter thinks we meant variable: "PIabc"
echo "And these will work:"
echo "1. The value of PI is $PI"
echo "2. The value of PI is ${PI}"
echo "3. The value of PI is" $PI
echo "And we can make a new string"
STR_A="Bob"
STR_B="Jane"
echo "${STR_A} + ${STR_B} equals Bob + Jane"
STR_C=${STR_A}" + "${STR_B}
echo "${STR_C} is the same as Bob + Jane too!"
echo "${STR_C} + ${PI}"
exit 0
--------------------------------
04_if.sh
#!/bin/bash
AGE=17
if [ ${AGE} -lt 18 ]; then
echo "You must be 18 or older to see this movie"
fi
--------------------------------------
05_if_else.sh
#!/bin/bash
AGE=9000
if [ ${AGE} -lt 18 ]; then
echo "You must be 18 or older to see this movie"
else
echo "You may see the movie!"
exit 1
fi
echo "This line will never get executed"
-----------------------------
06_if_elif.sh
#!/bin/bash
AGE=21
if [ ${AGE} -lt 18 ]; then
echo "You must be 18 or older to see this movie"
elif [ ${AGE} -eq 21 ]; then
echo "You may see the movie and get popcorn"
else
echo "You may see the movie!"
exit 1
fi
echo "This line might not get executed"
--------------------------------------
07_strings_evaluation.sh
#!/bin/bash
MY_NAME="John"
NAME_1="Bob"
NAME_2="Jane"
NAME_3="Sue"
Name_4="Kate"
if [ "${MY_NAME}" == "Ron" ]; then
echo "Ron is home from vacation"
elif [ "${MY_NAME}" != ${NAME_1}" && "${MY_NAME}" != ${NAME_2}" && "${MY_NAME}" == "John" ]; then
echo "John is home after some unnecessary AND logic"
elif [ "${MY_NAME}" == ${NAME_3}" || "${MY_NAME}" == ${NAME_4}" ]; then
echo "Looks like one of the ladies are home"
else
echo "Who is this stranger?"
fi
-----------------------------------------
08_nested_if.sh
#!/bin/bash
USER_AGE=18
AGE_LIMIT=18
NAME="Bob" # Change to your username if you want to execute the nested logic
HAS_NIGHTMARES="true"
if [ "${USER}" == "${NAME}" ]; then
if [ ${USER_AGE} -ge ${AGE_LIMIT} ]; then
if [ "${HAS_NIGHTMARES}" == "true" ]; then
echo "${USER} gets nightmares, and should not see the movie"
fi
fi
else
echo "Who is this?"
fi
----------------------------------------------
09_if_elif_else.sh
#!/bin/bash
VAR=10
# Multiple IF statements
if [ $VAR -eq 1 ]; then
echo "$VAR"
elif [ $VAR -eq 2]; then
echo "$VAR"
elif [ $VAR -eq 3]; then
echo "$VAR"
# .... to 10
else
echo "I am not looking to match this value"
fi
-----------------------------------------------
10_case.sh
#!/bin/bash
VAR=10 # Edit to 1 or 2 and re-run, after running the script as is.
case $VAR in
1)
echo "1"
;;
2)
echo "2"
;;
*)
echo "What is this var?"
exit 1
esac
---------------------------------------------------
11_for.sh
#!/bin/bash
FILES=( "file1" "file2" "file3" )
for ELEMENT in ${FILES[@]}
do
echo "${ELEMENT}"
done
echo "Echo\'d all the files"
-------------------------------
12_do_while.sh
#!/bin/bash
CTR=1
while [ ${CTR} -lt 9 ]
do
echo "CTR var: ${CTR}"
((CTR++)) # Increment the CTR variable by 1
done
echo "Finished"
---------------------------------
13_until.sh
#!/bin/bash
CTR=1
until [ ${CTR} -gt 9 ]
do
echo "CTR var: ${CTR}"
((CTR++)) # Increment the CTR variable by 1
done
echo "Finished"
---------------------------------------
14_function.sh
#!/bin/bash
FILES=( "file1" "file2" "file3" ) # This is a global variable
function create_file() {
local FNAME="${1}" # First parameter
local PERMISSIONS="${2}" # Second parameter
touch "${FNAME}"
chmod "${PERMISSIONS}" "${FNAME}"
ls -l "${FNAME}"
}
for ELEMENT in ${FILES[@]}
do
create_file "${ELEMENT}" "a+x"
done
echo "Created all the files with a function!"
exit 0
------------------------------------------
15_io_maker.sh
#!/bin/bash
source library.sh # You may need to include the path as it is relative
FNAME="my_test_file.txt"
create_file "${FNAME}"
delete_file "${FNAME}"
exit 0
--------------------------------------
16_library.sh
#!/bin/bash
function create_file() {
local FNAME=$1
touch "${FNAME}"
ls "${FNAME}" # If output doesn't return a value - file is missing
}
function delete_file() {
local FNAME=$1
rm "${FNAME}"
ls "${FNAME}" # If output doesn't return a value - file is missing
}
-------------------------------------------
17_return_codes_101.sh
#!/bin/bash
GLOBAL_RET=255
function my_function_global() {
ls /home/${USER}/.bashrc
GLOBAL_RET=$?
}
function my_function_return() {
ls /home/${USER}/.bashrc
return $?
}
function my_function_str() {
local UNAME=$1
local OUTPUT=""
if [ -e /home/${UNAME}/.bashrc ]; then
OUTPUT='FOUND IT'
else
OUTPUT='NOT FOUND'
fi
echo ${OUTPUT}
}
echo "Current ret: ${GLOBAL_RET}"
my_function_global "${USER}"
echo "Current ret after: ${GLOBAL_RET}"
GLOBAL_RET=255
echo "Current ret: ${GLOBAL_RET}"
my_function_return "${USER}"
GLOBAL_RET=$?
echo "Current ret after: ${GLOBAL_RET}"
# And for giggles, we can pass back output too!
GLOBAL_RET=""
echo "Current ret: ${GLOBAL_RET}"
GLOBAL_RET=$(my_function_str ${USER})
# You could also use GLOBAL_RET=`my_function_str ${USER}`
# Notice the back ticks "`"
echo "Current ret after: $GLOBAL_RET"
exit 0
----------------------------------
18_redirection.sh
#!/bin/sh
# Let's run a command and send all of the output to /dev/null
echo "No output?"
ls ~/fakefile.txt > /dev/null 2>&1
# Retrieve output from a piped command
echo "part 1"
HISTORY_TEXT=`cat ~/.bashrc | grep HIST`
echo "${HISTORY_TEXT}"
# Output the results to history.config
echo "part 2"
echo "${HISTORY_TEXT}" > "history.config"
# Re-direct history.config as input to the cat command
cat < history.config
# Append a string to history.config
echo "MY_VAR=1" >> history.config
echo "part 3 - using Tee"
# Neato.txt will contain the same information as the console
ls -la ~/fakefile.txt ~/ 2>&1 | tee neato.txt
-----------------------------------------
19_flags.sh
#!/bin/bash
HELP_STR="usage: $0 [-h] [-f] [-l] [--firstname[=]<value>] [--lastname[=]<value] [--help]"
# Notice hidden variables and other built-in Bash functionality
optspec=":flh-:"
while getopts "$optspec" optchar; do
case "${optchar}" in
-)
case "${OPTARG}" in
firstname)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
FIRSTNAME="${val}"
;;
lastname)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
LASTNAME="${val}"
;;
help)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
;;
*)
if [ "$OPTERR" = 1 ] && [ "${optspec:0:1}" != ":" ]; then
echo "Found an unknown option --${OPTARG}" >&2
fi
;;
esac;;
f)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
FIRSTNAME="${val}"
;;
l)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
LASTNAME="${val}"
;;
h)
echo "${HELP_STR}" >&2
exit 2
;;
*)
if [ "$OPTERR" != 1 ] || [ "${optspec:0:1}" = ":" ]; then
echo "Error parsing short flag: '-${OPTARG}'" >&2
exit 1
fi
;;
esac
done
# Do we have even one argument?
if [ -z "$1" ]; then
echo "${HELP_STR}" >&2
exit 2
fi
# Sanity check for both Firstname and Lastname
if [ -z "${FIRSTNAME}" ] || [ -z "${LASTNAME}" ]; then
echo "Both firstname and lastname are required!"
exit 3
fi
echo "Welcome ${FIRSTNAME} ${LASTNAME}!"
exit 0
Chapter 02
01_search.sh
#!/bin/bash
# Let's find all the files with the string "Packt"
DIRECTORY="www.packtpub.com/"
SEARCH_TERM="Packt"
# Can we use grep?
grep "${SEARCH_TERM}" ~/* > result1.txt 2&> /dev/null
# Recursive check
grep -r "${SEARCH_TERM}" "${DIRECTORY}" > result2.txt
# What if we want to check for multiple terms?
grep -r -e "${SEARCH_TERM}" -e "Publishing" "${DIRECTORY}" > result3.txt
# What about find?
find "${DIRECTORY}" -type f -print | xargs grep "${SEARCH_TERM}" > result4.txt
# What about find and looking for the string inside of a specific type of content?
find "${DIRECTORY}" -type f -name "*.xml" ! -name "*.css" -print | xargs grep "${SEARCH_TERM}" > result5.txt
# Can this also be achieved with wildcards and subshell?
grep "${SEARCH_TERM}" $(ls -R "${DIRECTORY}"*.{html,txt}) > result6.txt
RES=$?
if [ ${RES} -eq 0 ]; then
echo "We found results!"
else
echo "It broke - it shouldn't happen (Packt is everywhere)!"
fi
# Or for bonus points - a personal favorite
history | grep "ls" # This is really handy to find commands you ran yesterday!
# Aaaannnd the lesson is:
echo "We can do a lot with grep!"
exit 0
------------------------------------------
02_test.sh
#!/bin/bash
STR1='123 is a number, ABC is alphabetic & aBC123 is alphanumeric.'
echo "-------------------------------------------------"
# Want to find all of the files beginning with an uppercase character and end with .pdf?
ls * | grep [[:upper:]]*.pdf
echo "-------------------------------------------------"
# Just all of the directories in your current directory?
ls -l [[:upper:]]*
echo "-------------------------------------------------"
# How about all of the files we created with an expansion using the { } brackets?
ls [:lower:].test .
echo "-------------------------------------------------"
# Files with a specific extension OR two?
echo ${STR1} > test.txt
ls *.{test,txt}
echo "-------------------------------------------------"
# How about looking for specific punctuation and output on the same line
echo "${STR1}" | grep -o [[:punct:]] | xargs echo
echo "-------------------------------------------------"
# How about using groups and single character wildcards (only 5 results)
ls | grep -E "([[:upper:]])([[:digit:]])?.test?" | tail -n 5
exit 0
------------------------------------------------
03_mathexp.sh
#!/bin/bash
# Retrieve file system information and remove header
TARBALL="archive.tar.gz"
CURRENT_PART_ALL=$(df --output=size,avail,used /home -B 1M | tail -n1)
# Iterate through and fill array
IFS=' ' read -r -a array <<< $CURRENT_PART_ALL
# Retrieve the size of the contents of the tarball
COMPRESSED_SZ=$(tar tzvf "${TARBALL}" | sed 's/ \+/ /g' | cut -f3 -d' ' | sed '2,$s/^/+ /' | paste -sd' ' | bc)
echo "First inspection - is there enough space?"
if [ ${array[1]} -lt ${COMPRESSED_SZ} ]; then
echo "There is not enough space to decompress the binary"
exit 1
else
echo "Seems we have enough space on first inspection - continuing"
VAR=$((${array[0]} - ${array[2]}))
echo "Space left: ${VAR}"
fi
echo "Safety check - do we have at least double the space?"
CHECK=$((${array[1]}*2))
echo "Double - good question? $CHECK"
# Notice the use of the bc command?
RES=$(echo "$(mhelper "${array[1]}" "/" "2")>0" | bc -l)
if [[ "${RES}" == "1" ]]; then
echo "Sppppppaaaaccee (imagine zombies)"
fi
# We know that this will break! (Bash is driven by integers)
# if [ $(mhelper "${array[2]}" "/" "2") -gt 0 ]; then
#~ echo "Sppppppaaaaccee (imagine zombies) - syntax error"
# fi
# What if we tried this with Bash and a concept again referring to floats
# e.g., 0.5
# It would break
# if [ $((${array[2]} * 0.5 )) -gt 0 ]; then
# echo "Sppppppaaaaccee (imagine zombies) - syntax error"
# fi
# Then untar
tar xzvf ${TARBALL}
RES=$?
if [ ${RES} -ne 0 ]; then
echo "Error decompressing the tarball!"
exit 1
fi
echo "Decompressing tarball complete!"
exit 0
---------------------------------------------
04_script.sh
#!/bin/bash
# Index zero of VARIABLE is the char 'M' & is 14 bytes long
VARIABLE="My test string"
# ${VARIABLE:startingPosition:optionalLength}
echo ${VARIABLE:3:4}
---------------------------------------
05_builtin-str.sh
#!/bin/bash
# Let's play with variable arrays first using Bash's equivalent of substr
STR="1234567890asdfghjkl"
echo "first character ${STR:0:1}"echo "first three characters ${STR:0:3}"
echo "third character onwards ${STR: 3}"echo "forth to sixth character ${STR: 3: 3}"
echo "last character ${STR: -1}"
# Next, can we compare the alphabeticalness of strings?
STR2="abc"STR3="bcd"STR4="Bcd"
if [[ $STR2 < $STR3 ]]; then echo "STR2 is less than STR3"else echo "STR3 is greater than STR2"fi
# Does case have an effect? Yes, b is less than Bif [[ $STR3 < $STR4 ]]; then echo "STR3 is less than STR4"else echo "STR4 is greater than STR3"fi
---------------------------------------
06_builtin-strng.sh
#!/bin/bash
GB_CSV="testdata/garbage.csv"
EM_CSV="testdata/employees.csv"
# Let's strip the garbage out of the last lines in the CSV called garbage.csv
# Notice the forloop; there is a caveat
set IFS=,
set oldIFS = $IFS
readarray -t ARR < ${GB_CSV}
# How many rows do we have?
ARRY_ELEM=${#ARR[@]}
echo "We have ${ARRY_ELEM} rows in ${GB_CSV}"
# Let's strip the garbage - remove spaces
INC=0
for i in "${ARR[@]}"
do
:
res="${i//[^ ]}"
TMP_CNT="${#res}"
while [ ${TMP_CNT} -gt 0 ]; do
i=${i/, /,}
TMP_CNT=$[$TMP_CNT-1]
done
ARR[$INC]=$i
INC=$[$INC+1]
done
# Lets remove the last character from each line
INC=0
for i in "${ARR[@]}"
do
:
ARR[$INC]=${i::-1}
INC=$[$INC+1]
done
# Now, let's turn all of the characters into uppercase!
INC=0
for i in "${ARR[@]}"
do
:
ARR[$INC]=${i^^}
printf "%s" "${ARR[$INC]}"
INC=$[$INC+1]
echo
done
# In employees.csv update the first field to be prepended with a # character
set IFS=,
set oldIFS = $IFS
readarray -t ARR < ${EM_CSV}
# How many rows do we have?
ARRY_ELEM=${#ARR[@]}
echo;echo "We have ${ARRY_ELEM} rows in ${EM_CSV}"
# Let's add a # at the start of each line
INC=0
for i in "${ARR[@]}"
do
:
ARR[$INC]="#${i}"
printf "%s" "${ARR[$INC]}"
INC=$[$INC+1]
echo
done
# Bob had a name change, he wants to go by the name Robert - replace it!
echo
echo "Let's make Bob, Robert!"
INC=0
for i in "${ARR[@]}"
do
:
# We need to iterate through Bobs first
ARR[$INC]=${i/Bob/Robert}
printf "%s" "${ARR[$INC]}"
INC=$[$INC+1]
echo
done
# Can we delete the day of birth column?
# field to remove is 5 (but -4)
echo;echo "Lets remove the column: birthday (1-31)"
INC=0
COLUM_TO_REM=4
for i in "${ARR[@]}"
do
:
# Prepare to also parse the ARR element into another ARR for
# string manipulation
TMP_CNT=0
STR=""
IFS=',' read -ra ELEM_ARR <<< "$i"
for field in "${ELEM_ARR[@]}"; do
# Notice the multiple argument in an if statement
# AND that we catch the start of it once
if [ $TMP_CNT -ne 0 ] && [ $TMP_CNT -ne $COLUM_TO_REM ]; then
STR="${STR},${field}"
elif [ $TMP_CNT -eq 0 ]; then
STR="${STR}${field}"
fi
TMP_CNT=$[$TMP_CNT+1]
done
ARR[$INC]=$STR
echo "${ARR[$INC]}"
INC=$[$INC+1]
done
-------------------------------------
07_some-strs.sh
#!/bin/bash
STR="1234567890asdfghjkl"
echo -n "First character "; sed 's/.//2g' <<< $STR # where N = 2 (N +1)
echo -n "First three characters "; sed 's/.//4g' <<< $STR
echo -n "Third character onwards "; sed -r 's/.{3}//' <<< $STR
echo -n "Forth to sixth character "; sed -r 's/.{3}//;s/.//4g' <<< $STR
echo -n "Last character by itself "; sed 's/.*\(.$\)/\1/' <<< $STR
echo -n "Remove last character only "; sed 's/.$//' <<< $STR
-------------------------------------
08_more-strsng.sh
#!/bin/sh
GB_CSV="testdata/garbage.csv"
EM_CSV="testdata/employees.csv"
# Let's strip the garbage out of the last lines in the CSV called garbage.csv
# Notice the forloop; there is a caveat
set IFS=,
set oldIFS = $IFS
readarray -t ARR < ${GB_CSV}
# How many rows do we have?
ARRY_ELEM=${#ARR[@]}
echo "We have ${ARRY_ELEM} rows in ${GB_CSV}"
# Let's strip the garbage - remove spaces
INC=0
for i in "${ARR[@]}"
do
:
ARR[$INC]=$(echo $i | sed 's/ //g')
echo "${ARR[$INC]}"
INC=$[$INC+1]
done
# Remove the last character and make ALL upper case
INC=0
for i in "${ARR[@]}"
do
:
ARR[$INC]=$(echo $i | sed 's/.$//' | sed -e 's/.*/\U&/' )
echo "${ARR[$INC]}"
INC=$[$INC+1]
done
# Want to add a # at the beginning of each line?
set IFS=,
set oldIFS = $IFS
readarray -t ARR < ${EM_CSV}
INC=0
for i in "${ARR[@]}"
do
:
ARR[$INC]=$(sed -e 's/^/#/' <<< $i )
echo "${ARR[$INC]}"
INC=$[$INC+1]
done
# Sed can also be used on a per file basis too!
# Want to just strip Bob out and change his name to Robert by manipulating
# the file inplace?
sed -i 's/Bob/Robert/' ${EM_CSV}
sed -i 's/^/#/' ${EM_CSV} # In place, instead of on the data in the array
cat ${EM_CSV}
# Now lets remove the birthdate field from the files
# Starts to get more complex, but is done without a loop or using cut
awk 'BEGIN { FS=","; OFS="," } {$5="";gsub(",+",",",$0)}1' OFS=, ${EM_CSV}
-------------------------------------
09_echo-mayhem.sh
#!/bin/bash
# What about echo?
echo -n "Currently we have seen the command \"echo\" used before"
echo " in the previous script"
echo
echo -n "Can we also have \t tabs? \r\n\r\n?"
echo " NO, not yet!"
echo
echo -en "Can we also have \t tabs? \r\n\r\n?"
echo " YES, we can now! enable interpretation of backslash escapes"
echo "We can also have:"
echo -en '\xF0\x9F\x92\x80\n' # We can also use \0NNN for octal instead of \xFF for hexidecimal
echo "Check the man pages for more info ;)"
---------------------------------------------
10_printf-mayhem.sh
#!/bin/bash
export LC_NUMERIC="en_US.UTF-8"
printf "This is the same as echo -e with a new line (\\\n)\n"
DECIMAL=10.0
FLOAT=3.333333
FLOAT2=6.6666 # On purpose two missing values
printf "%s %.2f\n\n" "This is two decimal places: " ${DECIMAL}
printf "shall we align: \n\n %.3f %-.6f\n" ${FLOAT} ${FLOAT2}
printf " %10f %-6f\n" ${FLOAT} ${FLOAT2}
printf " %-10f %-6f\n" ${FLOAT} ${FLOAT2}
# Can we also print other things?
printf '%.0s-' {1..20}; printf "\n"
# How about we print the hex value and a char for each value in a string?
STR="No place like home!"
CNT=$(wc -c <<< $STR})
TMP_CNT=0
printf "Char Hex\n"
while [ ${TMP_CNT} -lt $[${CNT} -1] ]; do
printf "%-5s 0x%-2X\n" "${STR:$TMP_CNT:1}" "'${STR:$TMP_CNT:1}"
TMP_CNT=$[$TMP_CNT+1]
done
----------------------------------------------------
11_hellobonjour.sh
#!/bin/bash
. gettext.sh
function i_have() {
local COUNT=$1
###i18n: Please leave $COUNT as is
echo -e "\n\t" $(eval_ngettext "I have \$COUNT electronic device" "I have \$COUNT electronic devices" $COUNT)
}
echo $(gettext "Hello")
echo
echo $(gettext "What is your name?")
echo
###i18n: Please leave $USER as is
echo -e "\t" $(eval_gettext "My name is \$USER" )
echo
echo $(gettext "Do you have electronics?")
i_have 0
i_have 1
i_have 2
------------------------------------------------
12_translator.sh
#!/bin/bash./hellobonjour.sh
export TEXTDOMAIN="hellobonjour"export TEXTDOMAINDIR=`pwd`/locale
export LANGUAGE=fr./hellobonjour.sh
----------------------------------------------
13_files-extended.sh
#!/bin/bash
FILE_TO_TEST=""
function permissions() {
echo -e "\nWhat are our permissions on this $2?\n"
if [ -r $1 ]; then
echo -e "[R] Read"
fi
if [ -w $1 ]; then
echo -e "[W] Write"
fi
if [ -x $1 ]; then
echo -e "{X] Exec"
fi
}
function file_attributes() {
if [ ! -s $1 ]; then
echo "\"$1\" is empty"
else
FSIZE=$(stat --printf="%s" $1 2> /dev/null)
RES=$?
if [ $RES -eq 1 ]; then
return
else
echo "\"$1\" file size is: ${FSIZE}\""
fi
fi
if [ ! -O $1 ]; then
echo -e "${USER} is not the owner of \"$1\"\n"
fi
if [ ! -G $1 ]; then
echo -e "${USER} is not among the owning group(s) for \"$1\"\n"
fi
permissions $1 "file"
}
function dir_attributes() {
echo -e "Directory \"$1\" has children:\n\n"
ls $1 | xargs -0
permissions $1 "directory"
}
function checkout_file() {
FILE_TO_TEST=""
echo -en "\nWhat is the complete path of the file you want to inspect?\n # "
read FILE_TO_TEST
echo
# First, does it exist?
if [ ! -e ${FILE_TO_TEST} ]; then
echo "Error: \"${FILE_TO_TEST}\" does not exist!"
exit 1
fi
if [ -f ${FILE_TO_TEST} ]; then
file_attributes ${FILE_TO_TEST}
checkout_file
else
dir_attributes ${FILE_TO_TEST}
checkout_file
fi
}
echo "Welcome to the file attributes tester"
echo
echo "To exit, press CTRL + C"
checkout_file
--------------------------------------------
14_data-csv-to-xml.sh
#!/bin/bash
# Import template variables
source xml-parent.tpl
source word.tpl
OUTPUT_FILE="words.xml"
INPUT_FILE="words.csv"
DELIMITER=','
# Setup header
echo ${XML_HDR} > ${OUTPUT_FILE}
echo ${SRT_CONTR} >> ${OUTPUT_FILE}
# Enter content
echo ${ELM} | \
sed '{:q;N;s/\n/\\n/g;t q}'| \
awk \
'{ print "awk \x27 BEGIN{FS=\"'${DELIMITER}'\"}{print "$0"}\x27 '${INPUT_FILE}'"}' | \
sh >> ${OUTPUT_FILE}
# Append trailer
echo ${END_CONTR} >> ${OUTPUT_FILE}
cat ${OUTPUT_FILE}
--------------------------------------------
15_data-xml-to-json.sh
!#/bin/bash
INPUT_FILE"words.xml"
OUTPUT_FILE="words.json"
# Easy one line!
xml2json < ${INPUT_FILE} ${OUTPUT_FILE}
Chapter 03
01_whoami.sh
#!/bin/bash
VAR=$0
echo "I was ran as: $VAR"
------------------------------------------
02_mytree.sh
#!/bin/bash
CURRENT_LVL=0
function tab_creator() {
local X=0
local LVL=$1
local TABS="."
while [ $X -lt $LVL ]
do
# Concatonate strings
TABS="${TABS}${TABS}"
X=$[$X+1]
done
echo -en "$TABS"
}
function recursive_tree() {
local ENTRY=$1
for LEAF in ${ENTRY}/*
do
if [ -d $LEAF ];then
# If LEAF is a directory & not empty
TABS=$(tab_creator $CURRENT_LVL)
printf "%s\_ %s\n" "$TABS" "$LEAF"
CURRENT_LVL=$(( CURRENT_LVL + 1 ))
recursive_tree $LEAF $CURRENT_LVL
CURRENT_LVL=$(( CURRENT_LVL - 1 ))
elif [ -f $LEAF ];then
# Print only the bar and not the backwards slash
# And only if a file
TABS=$(tab_creator $CURRENT_LVL)
printf "%s|_%s\n" "$TABS" "$LEAF"
continue
fi
done
}
PARENTDIR=$1
recursive_tree $PARENTDIR 1
------------------------------------------
03_dsetmkr.sh
$ #!/bin/bash
BDIR="files_galore"
rm -rf ${BDIR}
mkdir -p ${BDIR}
touch $BDIR/file1; echo "1111111111111111111111111111111" >
$BDIR/file1;
touch $BDIR/file2; echo "2222222222222222222222222222222" > $BDIR/file2;
touch $BDIR/file3; echo "3333333333333333333333333333333" > $BDIR/file3;
touch $BDIR/file4; echo "4444444444444444444444444444444" > $BDIR/file4;
touch $BDIR/file5; echo "4444444444444444444444444444444" > $BDIR/file5;
touch $BDIR/sameas5; echo "4444444444444444444444444444444" > $BDIR/sameas5;
touch $BDIR/sameas1; echo "1111111111111111111111111111111" > $BDIR/sameas1;
------------------------------------------
04_file-deduplicator.sh
#!/bin/bash
declare -a FILE_ARRAY=()
function add_file() {
# echo $2 $1
local NUM_OR_ELEMENTS=${#FILE_ARRAY[@]}
FILE_ARRAY[$NUM_OR_ELEMENTS+1]=$1
}
function del_file() {
rm "$1" 2>/dev/null
}
function srch_file() {
local NEW="$1"
local SUM="0"
if [ -f $NEW ] && [ ${#FILE_ARRAY[@]} -eq 0 ]; then
# Edge case - can you guess why?
echo $(sha512sum ${NEW} | awk -F' ' '{print $1}')
return
fi
for ELEMENT in ${FILE_ARRAY[@]}
do
SUM=$(sha512sum ${NEW} | awk -F' ' '{print $1}')
if [ "${SUM}" == "${ELEMENT}" ]; then
# Return 1 if ${NEW} is a know file in the array
return "1"
else
# Continue the loop
continue
fi
done
# Return 2 if ${NEW} is a file && is not found
echo "${SUM}"
}
function begin_search_and_deduplication() {
local DIR_TO_SRCH="$1"
for FILE in ${DIR_TO_SRCH}/*
do
RET=$(srch_file ${FILE})
if [[ "${RET}" == "" ]]; then
del_file $FILE
continue
elif [[ "${RET}" == "0" ]]; then
# Duplicate file - delete
continue
elif [[ "${RET}" == "1" ]]; then
continue
else
# Add file and continue
add_file "${RET}" "${FILE}"
continue
fi
done
}
function dump_array() {
local ARR=$1
local SIZE=${#FILE_ARRAY[@]}
# We start at 1 here because our first element is installed at 1 and not 0
# We leave it as an exercise to the reader
for (( i=1; i <= ${SIZE}; i++ ));
do
echo "#$i " ${FILE_ARRAY[$i]}
done
}
echo "Enter directory name to being searching and deduplicating:"
echo "Press [ENTER] when ready"
echo
read DIR
begin_search_and_deduplication "${DIR}"
dump_array
------------------------------------------
05_file-splitter.sh
--------
#!/bin/bash
FNAME=""
LEN=10
TYPE="line"
OPT_ERROR=0
set -f
function determine_type_of_file() {
local FILE="$1"
file -b "${FILE}" | grep "ASCII text" > /dev/null
RES=$?
if [ $RES -eq 0 ]; then
echo "ASCII file - continuing"
else
echo "Not an ASCII file, perhaps it is Binary?"
fi
}
function write_to_file() {
local B_FNAME="$1"
echo -en "${BUFFER}" > "${B_FNAME}.${F_CNT}"
echo "Wrote buffer to file: ${B_FNAME}.${F_CNT}"
}
function read_and_split_by_line() {
local B_FNAME="$1"
local L_CNT=0
local F_CNT=0
local BUFFER=""
# Read with the -r flag
while IFS= read -r LINE;
do
BUFFER+="${LINE}\n"
L_CNT=$((L_CNT+1))
# Lines are set to 10 by default, but should be set to something more
# efficent for IO
if [ ${L_CNT} -eq ${LEN} ]; then
F_CNT=$((F_CNT+1))
write_to_file "${B_FNAME}" ${F_CNT} "${BUFFER}"
L_CNT=0
BUFFER=""
fi
done < ${B_FNAME}
# If we read everything, we should write what remains in the buffer!
if [ "${BUFFER}" != "" ]; then
F_CNT=$((F_CNT+1))
write_to_file "${B_FNAME}" ${F_CNT} "${BUFFER}"
fi
}
function read_and_write_by_size() {
local B_FNAME="$1"
local L_CNT=0
local F_CNT=0
local BUFFER=""
PAGESIZE=$(getconf PAGESIZE)
# Read with the -n flag
while IFS= read -n ${PAGESIZE} LINE;
do
BUFFER+="${LINE}"
L_CNT=$((L_CNT+1))
# Len should reflect an optimal bloc ksize or something for
# efficient IO
if [ $L_CNT -eq ${LEN} ]; then
F_CNT=$((F_CNT+1))
write_to_file "${B_FNAME}" ${F_CNT} "${BUFFER}"
L_CNT=0
BUFFER=""
fi
done < ${B_FNAME}
# If we read everything, we should write what remains in the buffer!
if [ "${BUFFER}" != "" ]; then
F_CNT=$((F_CNT+1))
write_to_file "${B_FNAME}" ${F_CNT} "${BUFFER}"
fi
}
# Add some extra fun to the script
while getopts ":i:l:t:" opt; do
case $opt in
i)
FNAME="$OPTARG"
if [ ! -e $FNAME ] && [ ! -f $FNAME ]; then
echo "ERROR: Input file parameter does not exit, or is not a file"
OPT_ERROR+=1
fi
;;
t)
TYPE="$OPTARG"
if [[ "${TYPE}" != "line" && "${TYPE}" != "size" ]]; then
echo "ERROR: -t must be set to line or size"
OPT_ERROR+=1
fi
;;
l)
LEN="$OPTARG"
if [ $LEN -le 0 ]; then
echo "ERROR: -l must be greater than 0"
OPT_ERROR+=1
fi
;;
\?)
echo "Invalid option: -$OPTARG" >&2
OPT_ERROR+=1
;;
:)
echo "Option -$OPTARG requires an argument." >&2
OPT_ERROR+=1
;;
esac
done
# A bit more sanity checking before continuing
if [ "${FNAME}" == "" ]; then
echo "ERROR: -i input parameter required; filename "
OPT_ERROR+=1
fi
# Exit if we have any errors, otherwise continue to run the splitting
# functionality :)
if [ ${OPT_ERROR} -ne 0 ]; then
exit 1
fi
# Make sure the file is ASCII text - sanity check
determine_type_of_file "${FNAME}"
# And begin!
if [ "${TYPE}" == "size" ]; then
read_and_write_by_size "${FNAME}"
else
read_and_split_by_line "${FNAME}"
fi
# Done!
exit 0
----------------------------------
06_file-joiner.sh
#!/bin/bash
INAME=""
ONAME=""
FNAME=""
WHERE=""
OPT_ERROR=0
TMPFILE1=$(mktemp)
function determine_type_of_file() {
local FILE="$1"
file -b "${FILE}" | grep "ASCII text" > /dev/null
RES=$?
if [ $RES -eq 0 ]; then
echo "ASCII file - continuing"
else
echo "Not an ASCII file, perhaps it is Binary?"
fi
}
function open_input_and_insert() {
local INPUT="$1"
local N_BLOCK="$2"
local FINAL_OUTPUT="$3"
local PTR=$4
local BUFFER=""
local CTR=0
# If the variable for where is not set, just append it.
if [[ "${PTR}" == "" ]]; then
cat "${INPUT}" "${N_BLOCK}" > "${FINAL_OUTPUT}"
return
fi
while IFS= read LINE
do
if [ ${CTR} == ${PTR} ]; then
cat "${N_BLOCK}" >> ${TMPFILE1}
if [ ${CTR} == 0 ]; then
echo -ne "${LINE}\n" >> ${TMPFILE1}
fi
else
echo -ne "${LINE}\n" >> ${TMPFILE1}
fi
CTR=$((CTR + 1))
done < ${INPUT}
mv ${TMPFILE1} ${FINAL_OUTPUT}
}
# Add some extra fun to the script
while getopts ":i:o:w:f:" opt; do
case $opt in
i)
INAME="$OPTARG"
if [ ! -e $INAME ] && [ ! -f $INAME ]; then
echo "ERROR: Input file parameter does not exit, or is not a file"
OPT_ERROR+=1
fi
;;
o)
ONAME="$OPTARG"
;;
f)
FNAME="$OPTARG"
;;
w)
WHERE="$OPTARG"
if [ "${WHERE}" -lt "0" ]; then
echo "ERROR: -w must be greater than 0"
OPT_ERROR+=1
fi
;;
\?)
echo "Invalid option: -$OPTARG" >&2
OPT_ERROR+=1
;;
:)
echo "Option -$OPTARG requires an argument." >&2
OPT_ERROR+=1
;;
esac
done
# A bit more sanity checking before continuing
if [ "${INAME}" == "" ]; then
echo "ERROR: -i input parameter required; filename "
OPT_ERROR+=1
fi
if [ "${ONAME}" == "" ]; then
echo "ERROR: -o input parameter required; filename "
OPT_ERROR+=1
fi
if [ "${FNAME}" == "" ]; then
echo "ERROR: -f FINAL file parameter required; filename "
OPT_ERROR+=1
fi
# Exit if we have any errors, otherwise continue to run the splitting
# functionality :)
if [ ${OPT_ERROR} -ne 0 ]; then
# Cleanup temporary file
unlink ${TMPFILE1}
exit 1
fi
# Make sure the file is ASCII text - sanity check
determine_type_of_file "${INAME}"
# And begin!
open_input_and_insert "${INAME}" "${ONAME}" "${FNAME}" ${WHERE}
# Done !
exit 0
--------------------------------------------
07_data-maker.sh
#!/bin/bash
N_FILES=3
TYPE=binary
DIRECTORY="qa-data"
NAME="garbage"
EXT=".bin"
UNIT="M"
RANDOM=$$
TMP_FILE="/tmp/tmp.datamaker.sh"
function get_random_number() {
SEED=$(($(date +%s%N)/100000))
RANDOM=$SEED
# Sleep is needed to make sure that the next time rnadom is ran, everything is good.
sleep 3
local STEP=$1
local VARIANCE=$2
local UPPER=$3
local LOWER=$VARIANCE
local ARR;
INC=0
for N in $( seq ${LOWER} ${STEP} ${UPPER} );
do
ARR[$INC]=$N
INC=$(($INC+1))
done
RAND=$[$RANDOM % ${#ARR[@]}]
echo $RAND
}
function create_binary_files(){
EXT=".bin"
local NUM_FILES=$1
local LOWER=$2
local UPPER=$3
local COUNTER=0
while [ $COUNTER -lt ${NUM_FILES} ]; do
R_SIZE=$(get_random_number 1 ${LOWER} ${UPPER})
echo "Creating file... please wait..."
dd if=/dev/zero of="${DIRECTORY}/${NAME}${COUNTER}${EXT}" bs="${R_SIZE}${UNIT}" count=1 2> /dev/null
let COUNTER=COUNTER+1
done
}
function create_text_files() {
EXT=".txt"
local NUM_FILES=$1
local VARIANCE=$2
local MIDDLE=$(($3 / 4))
local UPPER=$3
local LOWER=$(($MIDDLE - $VARIANCE))
if [ $LOWER -lt 0 ]; then
LOWER=$(($LOWER * -1))
LOWER=$(($LOWER + 1))
fi
local TRUE=0
local COUNTER=0
while [ $COUNTER -lt ${NUM_FILES} ]; do
END=$(get_random_number 1 ${VARIANCE} ${UPPER})
START=$(get_random_number 1 ${VARIANCE} ${LOWER})
TRUE=0
while [ $TRUE -eq 0 ]; do
if [ $END -gt $START ]; then
TRUE=1
else
echo "Generating random values... please wait..."
END=$(get_random_number 1 ${VARIANCE} ${UPPER})
continue
fi
done
echo "Creating file... please wait..."
dd if="${TMP_FILE}" of="${DIRECTORY}/${NAME}${COUNTER}${EXT}" seek=${START} bs=1 count=$((${END} -${START})) 2> /dev/null
let COUNTER=COUNTER+1
done
}
# Add some extra fun to the script
OPT_ERROR=0
while getopts ":t:n:l:u:" opt; do
case $opt in
t)
TYPE="$OPTARG"
if [[ "${TYPE}" != "binary" && "${TYPE}" != "text" ]]; then
echo "ERROR: -t must be set to line or size"
OPT_ERROR+=1
fi
;;
n)
N_FILES="$OPTARG"
if [ $N_FILES -le 0 ]; then
echo "ERROR: -l must be greater than 0"
OPT_ERROR+=1
fi
;;
l)
LOWER="$OPTARG"
if [ $LOWER -le 0 ]; then
echo "ERROR: -l must be greater than 0"
OPT_ERROR+=1
fi
;;
u)
UPPER="$OPTARG"
if [ $UPPER -le 0 ]; then
echo "ERROR: -l must be greater than 0"
OPT_ERROR+=1
fi
;;
\?)
echo "Invalid option: -$OPTARG" >&2
OPT_ERROR+=1
;;
:)
echo "Option -$OPTARG requires an argument." >&2
OPT_ERROR+=1
;;
esac
done
# Exit if we have any errors, otherwise continue to run the splitting
# functionality :)
if [ ${OPT_ERROR} -ne 0 ]; then
exit 1
fi
case "$TYPE" in
binary)
create_binary_files $N_FILES $LOWER $UPPER
;;
text)
create_text_files $N_FILES $LOWER $UPPER
;;
:)
echo "Unknown type of operaiton"
;;
esac
echo "DONE!"
exit 0
chapter 04
01_recursive_read_input.sh
#!/bin/bash
function recursive_func() {
echo -n "Press anything to continue loop "
read input recursive_func
}
recursive_funcexit 0
--------------------------------------------
02_loop_for_input.sh
#!/bin/bash
for (( ; ; ))
do
echo "Shall run for ever"
sleep 1
done
exit 0
--------------------------------------------
03_loop_while_input.sh
!/bin/bash
EXIT_PLEASE=0
while : # Notice no conditions?
do
echo "Pres CTRL+C to stop..."
sleep 1
if [ $EXIT_PLEASE != 0 ]; then
break
fi
done
exit 0
--------------------------------------------
04_loop_until_input.sh
#!/bin/bash
EXIT_PLEASE=0
until [ $EXIT_PLEASE != 0 ] # EXIT_PLEASE is set to 0, until will never be satisfied
do
echo "Pres CTRL+C to stop..."
sleep 1
done
exit 0
--------------------------------------------
05_loop_and_print.sh
#!/bin/bash
EXIT_PLEASE=0
INC=0
until [ ${EXIT_PLEASE} != 0 ] # EXIT_PLEASE is set to 0, until will never be satisfied
do
echo "Boo $INC" > /dev/null
INC=$((INC + 1))
sleep 1
done
exit 0
--------------------------------------------
06_bad_input.sh
#!/bin/bash
FILE_NAME=$1
echo $FILE_NAME
ls $FILE_NAME
--------------------------------------------
07_better_input.sh
#!/bin/bash
FILE_NAME=$1
# first, strip underscores
FILE_NAME_CLEAN=${FILE_NAME//_/}
FILE_NAME_CLEAN=$(sed 's/..//g' <<< ${FILE_NAME_CLEAN})
# next, replace spaces with underscores
FILE_NAME_CLEAN=${FILE_NAME_CLEAN// /_}
# now, clean out anything that's not alphanumeric or an underscore
FILE_NAME_CLEAN=${FILE_NAME_CLEAN//[^a-zA-Z0-9_.]/}
# here you should check to see if the file exists before running the command
ls "${FILE_NAME_CLEAN}"
--------------------------------------------
08_validate_email.sh
#!/bin/bash
EMAIL=$1
echo "${EMAIL}" | grep '^[a-zA-Z0-9._]*@[a-zA-Z0-9]*\.[a-zA-Z0-9]*$' >/dev/null
RES=$?
if [ $RES -ne 1 ]; then
echo "${EMAIL} is valid"
else
echo "${EMAIL} is NOT valid"
fi
--------------------------------------------
09_validate_ip.sh
#!/bin/bash
IP_ADDR=$1
IFS=.
if echo "$IP_ADDR" | { read octet1 octet2 octet3 octet4 extra;
[[ "$octet1" == *[[:digit:]]* ]] &&
test "$octet1" -ge 0 && test "$octet1" -le 255 &&
[[ "$octet2" == *[[:digit:]]* ]] &&
test "$octet2" -ge 0 && test "$octet2" -le 255 &&
[[ "$octet3" == *[[:digit:]]* ]] &&
test "$octet3" -ge 0 && test "$octet3" -le 255 &&
[[ "$octet4" == *[[:digit:]]* ]] &&
test "$octet4" -ge 0 && test "$octet4" -le 255 &&
test -z "$extra" 2> /dev/null; }; then
echo "${IP_ADDR} is valid"
else
echo "${IP_ADDR} is NOT valid"
fi
--------------------------------------------
10_select_menu.sh
#!/bin/bash
TITLE="Select file menu"
PROMPT="Pick a task:"
OPTIONS=("list" "delete" "modify" "create")
function list_files() {
PS3="Choose a file from the list or type \"back\" to go back to the main: "
select OPT in *; do
if [[ $REPLY -le ${#OPT[@]} ]]; then
if [[ "$REPLY" == "back" ]]; then
return
fi
echo "$OPT was selected"
else
list_files
fi
done
}
function main_menu() {
echo "${TITLE}"
PS3="${PROMPT} "
select OPT in "${OPTIONS[@]}" "quit"; do
case "$REPLY" in
1 )
# List
list_files
main_menu # Recursive call to regenerate the menu
;;
2 )
echo "not used"
;;
3 )
echo "not used"
;;
4 )
echo "not used"
;;
$(( ${#OPTIONS[@]}+1 )) ) echo "Exiting!"; break;;
*) echo "Invalid option. Try another one.";continue;;
esac
done
}
main_menu # Enter recursive loop
--------------------------------------------
11_mytrap.sh
#!/bin/bash
function setup() {
trap "cleanup" SIGINT SIGTERM
echo "PID of script is $$"
}
function cleanup() {
echo "cleaning up"
exit 1
}
setup
# Loop forever with a noop (:)
while :
do
sleep 1
done
--------------------------------------------
12_mylock.sh
#!/bin.bash
LOCKFILE="/tmp/mylock"
function setup() {
# $$ will provide the process ID
TMPFILE="${LOCKFILE}.$$"
echo "$$" > "${TMPFILE}"
# Now we use hard links for atomic file operations
if ln "${TMPFILE}" "${LOCKFILE}" 2>&- ; then
echo "Created tmp lock"
else
echo "Locked by" $(<$LOCKFILE)
rm "${TMPFILE}"
exit 1
fi
trap "rm ${TMPFILE} ${LOCKFILE}" SIGINT SIGTERM SIGKILL
}
setup
echo "Door was left unlocked"
exit 0
--------------------------------------------
13_mytimeout.sh
#!/bin/bash
SUBPID=0
function func_timer() {
trap "clean_up" SIGALRM
sleep $1& wait
kill -s SIGALRM $$
}
function clean_up() {
trap - ALRM
kill -s SIGALRM $SUBPID
kill $! 2>/dev/null
}
# Call function for timer & notice we record the job's PID
func_timer $1& SUBPID=$!
# Shift the parameters to ignore $0 and $1
shift
# Setup the trap for signals SIGALRM and SIGINT
trap "clean_up" ALRM INT
# Execute the command passed and then wait for a signal
"$@" & wait $!
# kill the running subpid and wait before exit
kill -ALRM $SUBPID
wait $SUBPID
exit 0
--------------------------------------------
14_master.sh
#!/bin/bash
FIFO_FILE=/tmp/WORK_QUEUE_FIFO
mkfifo "${FIFO_FILE}"
NUM_WORKERS=5
I=0
while [ $I -lt $NUM_WORKERS ]; do
bash worker.sh "$I" &
I=$((I+1))
done
I=0
while [ $I -lt $NUM_WORKERS ]; do
echo "$I" > "${FIFO_FILE}"
I=$((I+1))
done
sleep 5
rm -rf "${FIFO_FILE}"
exit 0
--------------------------------------------
15_worker.sh
#!/bin/bash
FIFO_FILE=/tmp/WORK_QUEUE_FIFO
BUFFER=""
echo "WORKER started: $1"
while :
do
read BUFFER < "${FIFO_FILE}"
if [ "${BUFFER}" != "" ]; then
echo "Worker received: $BUFFER"
exit 1
fi
done
exit 0
--------------------------------------------
16_myscript.sh
#!/bin/bash
for (( ; ; ))
do
sleep 1
done
chapter 05
01_server_uptime.sh
serveruptime=`uptime | awk '{print $3,$4}'| sed 's/,//'| grep "day"`;
if [[ -z "$serveruptime" ]]; then
serveruptime=`uptime | awk '{print $3}'| sed 's/,//'`;
echo $serveruptime
else
:;
fi;
--------------------------------------------
02_cpu_info.sh
for cpus in `dmidecode -t4|awk '/Handle / {print $2}'`; do
echo `dmidecode -t4|sed '/Flags/,/Version/d'|egrep -A20 "Handle ${cpus}"|grep -m 1 "Socket Designation"|grep -o '.\{0,0\}:.\{0,18\}'|tr -d '\:| '`;
echo `dmidecode -t4|sed '/Flags/,/Version/d'|egrep -A20 "Handle ${cpus}"|grep -m 1 "Family"|grep -o '.\{0,0\}:.\{0,18\}'|tr -d '\:| '`;
echo `dmidecode -t4|sed '/Flags/,/Version/d'|egrep -A20 "Handle ${cpus}"|grep -m 1 "Manufacturer"|grep -o '.\{0,0\}:.\{0,18\}'|tr -d '\:| '`;
echo `dmidecode -t4|sed '/Flags/,/Version/d'|egrep -A20 "Handle ${cpus}"|grep -m 1 "Current Speed"|grep -o '.\{0,0\}:.\{0,18\}'|tr -d '\:| '`;
echo `dmidecode -t4|sed '/Flags/,/Version/d'|egrep -A20 "Handle ${cpus}"|grep -m 1 "Voltage"|grep -o '.\{0,0\}:.\{0,18\}'|tr -d '\:| '`;
echo `dmidecode -t4|sed '/Flags/,/Version/d'|egrep -A20 "Handle ${cpus}"|grep -m 1 "Core Count"|grep -o '.\{0,0\}:.\{0,18\}'|tr -d '\:| '`;
done
--------------------------------------------
03_test_ipv4.sh
if ping -q -c 1 -W 1 8.8.8.8 >/dev/null; then
echo "IPv4 is up"
else
echo "IPv4 is down"
fi
--------------------------------------------
04_test_ip_dns.sh
if ping -q -c 1 -W 1 google.com >/dev/null; then
echo "The network is up"
else
echo "The network is down"
fi
--------------------------------------------
05_test_web.sh
case "$(curl -s --max-time 2 -I http://google.com | sed 's/^[^ ]* *\([0-9]\).*/\1/; 1q')" in
[23]) echo "HTTP connectivity is up";;
5) echo "The web proxy won't let us through";;
*) echo "The network is down or very slow";;
esac
--------------------------------------------
06_wifi_conn.sh
#!/bin/sh
## usage:
## sudo ./wpa2-wifi-connect.sh <ssid> <pass>
ifdown wlan0
# build the interfaces file that will point to the file that holds our configuration
rm /etc/network/interfaces
touch /etc/network/interfaces
echo 'auto lo' >> /etc/network/interfaces
echo 'iface lo inet loopback' >> /etc/network/interfaces
echo 'iface eth0 inet dhcp' >> /etc/network/interfaces
echo 'allow-hotplug wlan0' >> /etc/network/interfaces
echo 'iface wlan0 inet manual' >> /etc/network/interfaces
echo 'wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf' >> /etc/network/interfaces
echo 'iface default inet dhcp' >> /etc/network/interfaces
# build the supplicant file that holds our configuration
rm /etc/wpa_supplicant/wpa_supplicant.conf
touch /etc/wpa_supplicant/wpa_supplicant.conf
echo 'ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev' >> /etc/wpa_supplicant/wpa_supplicant.conf
echo 'update_config=1' >> /etc/wpa_supplicant/wpa_supplicant.conf
wpa_passphrase $1 $2 >> /etc/wpa_supplicant/wpa_supplicant.conf
ifup wlan0
--------------------------------------------
07_inotify_example.sh
#! /bin/bash
folder=~/Desktop/abc
cdate=$(date +"%Y-%m-%d-%H:%M")
inotifywait -m -q -e create -r --format '%:e %w%f' $folder | while read file
do
mv ~/Desktop/abc/output.txt ~/Desktop/Old_abc/${cdate}-output.txt
done
--------------------------------------------
08_mysql_version.sh
#!/bin/bash
mysql -u root -pTraining2@^ <<MY_QUERY
SELECT VERSION();
MY_QUERY
--------------------------------------------
09_create_database.sh
#!/bin/bash
mysql -u root -pTraining2@^ <<MY_QUERY
create database testdb;
MY_QUERY
--------------------------------------------
10_add_user.sh
#!/bin/bash
#set -x
MY_INPUT='/home/mansijoshi/Desktop'
declare -a A_SURNAME
declare -a A_NAME
declare -a A_USERNAME
declare -a A_DEPARTMENT
declare -a A_PASSWORD
while IFS=, read -r COL1 COL2 COL3 COL4 COL5 TRASH;
do
A_SURNAME+=("$COL1")
A_NAME+=("$COL2")
A_USERNAME+=("$COL3")
A_DEPARTMENT+=("$COL4")
A_PASSWORD+=("$COL5")
done <"$MY_INPUT"
for index in "${!A_USERNAME[@]}"; do
useradd -g "${A_DEPARTMENT[$index]}" -d "/home/${A_USERNAME[$index]}" -s /bin/bash -p "$(echo "${A_PASSWORD[$index]}" | openssl passwd -1 -stdin)"
"${A_USERNAME[$index]}"
done
chapter 06
01_create_alarm.sh
#!/bin/bash
declare -i H
declare -i M
declare -i cur_H
declare -i cur_M
declare -i min_left
declare -i hour_left
echo -e "What time do you Wake Up?"
read H
echo -e "and Minutes?"
read M
cur_H=`date +%H`
cur_M=`date +%M`
echo "You Selected "
echo "$H:$M"
echo -e "\nIt is Currently $cur_H:$cur_M"
if [ $cur_H -lt $H ]; then
hour_left=`expr $H - $cur_H`
echo "$H - $cur_H means You Have: $hour_left hours still"
fi
if [ $cur_H -gt $H ]; then
hour_left=`expr $cur_H - $H`
echo -e "\n$cur_H - $H means you have $hour_left hours left \n"
fi
if [ $cur_H == $H ]; then
hour_left=0
echo -e "Taking a nap?\n"
fi
if [ $cur_M -lt $M ]; then
min_left=`expr $M - $cur_M`
echo -e "$M -$cur_M you have: $min_left minutes still"
fi
if [ $cur_M -gt $M ]; then
min_left=`expr $cur_M - $M`
echo -e "$cur_M - $M you have $min_left minutes left \n"
fi
if [ $cur_M == $M ]; then
min_left=0
echo -e "and no minutes\n"
fi
echo -e "Sleeping for $hour_left hours and $min_left minutes \n"
sleep $hour_left\h
sleep $min_left\m
mplayer ~/.alarm/alarm.mp3
--------------------------------------------
02_yes_no.sh
dialog --yesno "Do you wish to continue?" 0 0
a=$?
if [ "${a}" == "0" ]; then
echo Yes
else
echo No
fi
--------------------------------------------
03_calendar_dialog.sh
dialog --calendar "Select a date... " 0 0 1 1 2018
val=$?
--------------------------------------------
04_checklist_dialog.sh
dialog --stdout --checklist "Enable the account options you want:" 10 40 3 \
1 "Home directory" on \
2 "Signature file" off \
3 "Simple password" off
--------------------------------------------
05_raise_border.sh
convert -raise 5x5 mountain.png mountain-raised.png
--------------------------------------------
06_integrity_check.sh
#!/bin/bash
E_DIR_NOMATCH=50
E_BAD_DBFILE=51
dbfile=Filerec.sha256
# storing records.
set_up_database ()
{
echo ""$directory"" > "$dbfile"
# Write directory name to first line of file.
sha256sum "$directory"/* >> "$dbfile"
# Append sha256 checksums and filenames.
}
check_database ()
{
local n=0
local filename
local checksum
if [ ! -r "$dbfile" ]
then
echo "Unable to read checksum database file!"
exit $E_BAD_DBFILE
fi
while read rec[n]
do
directory_checked="${rec[0]}"
if [ "$directory_checked" != "$directory" ]
then
echo "Directories do not match up!"
# Tried to use file for a different directory.
exit $E_DIR_NOMATCH
fi
if [ "$n" -gt 0 ]
then
filename[n]=$( echo ${rec[$n]} | awk '{ print $2 }' )
# sha256sum writes recs backwards,
#+ checksum first, then filename.
checksum[n]=$( sha256sum "${filename[n]}" )
if [ "${rec[n]}" = "${checksum[n]}" ]
then
echo "${filename[n]} unchanged."
else
echo "${filename[n]} : CHECKSUM ERROR!"
fi
fi
let "n+=1"
done <"$dbfile" # Read from checksum database file.
}
if [ -z "$1" ]
then
directory="$PWD" # If not specified,
else
directory="$1"
fi
clear
if [ ! -r "$dbfile" ]
then
echo "Setting up database file, \""$directory"/"$dbfile"\".";
echo
set_up_database
fi
check_database
echo
exit 0
--------------------------------------------
07_timezones.sh
TZ=":Antarctica/Casey" date
TZ=":Atlantic/Bermuda" date
TZ=":Asia/Calcutta" date
TZ=":Europe/Amsterdam" date
--------------------------------------------
Filerec.md5
/home/student/chapter6
627aba5bbcb3baeb5d2b93c9253366f7 /home/student/chapter6/01_create_alarm.sh
92f6dc226c3cd2b6022dc3ba67a8ddc7 /home/student/chapter6/02_yes_no.sh
0c198b56863bf9803cf6b42dba62a6ea /home/student/chapter6/03_calendar_dialog.sh
c40d3d7f38961ca413de45991816aa76 /home/student/chapter6/04_checklist_dialog.sh
f5c58e4f38384f667456537d63d60f0c /home/student/chapter6/05_raise_border.sh
ad50042115b652d5b258923fbb48ea6e /home/student/chapter6/06_integrity_check.sh
f7b122de4b196b61273d3960d2430085 /home/student/chapter6/07_timezones.sh
f19d1dc2d81a9f174ecfdbd1a75759d4 /home/student/chapter6/1.sh
846b5af8c963284c6099e342fcf74d21 /home/student/chapter6/Filerec.md5
d36b9816e2e23ecd869286d671ea1061 /home/student/chapter6/Filerec.sha256
--------------------------------------------
Filerec.sha256
home/student/chapter6
2eff154a81ce00e9200bd640e4f0d9867a1c93d2506f05c4988b3edf7e5fe989
/home/student/chapter6/01_create_alarm.sh
43fc3c1c02ccf66fcd4f9fc86c1bf030cecbbfe52924d641b0f3da8bf559768e /home/student/chapter6/02_yes_no.sh
502a23a731837c02b6c89ae69433d2de54d047ebd76df4b0331812e5823bff05 /home/student/chapter6/03_calendar_dialog.sh
2e239622ff93691740cd0f89daadc1d9904bcb996d4d3520a658fa4bfb322dde
/home/student/chapter6/04_checklist_dialog.sh
4152530fbeb2705973f53f4e1dce6410b05169c3b1570f1f544eead0a7082e64
/home/student/chapter6/05_raise_border.sh
306de4d77df8f106a6d9f76119196bf0ec7f2901e775e6a40abe3c4a1e016534
/home/student/chapter6/06_integrity_check.sh
3af02d2d2090cc2c95054aa9b5206231b7cff3f9ee0bb9751253cae833336f2f
/home/student/chapter6/07_timezones.sh
07caed801936b890e913c9fbbf052cd7323f3232a1674b2fefe4b0b6994b9939 /home/student/chapter6/Filerec.sha256
chapter 07
--------------------------------------------
01_scrap_contents.sh
$ mkdir -p data
$ cd data
$ wget -q -r -l5 -x 5 https://imdb.com
$ cd ..
$ grep -r -Po -h '(?<=href=")[^"]*' data/ > links.csv
$ grep "^http" links.csv > links_filtered.csv
$ sort -u links_filtered.csv > links_final.csv
$ rm -rf data links.csv links_filtered.csv
--------------------------------------------
02_logging_bot.sh
#!/bin/bash
nick="blb$$"
channel=testchannel
server=irc.freenode.net
config=/tmp/irclog
[ -n "$1" ] && channel=$1
[ -n "$2" ] && server=$2
config="${config}_${channel}"
echo "NICK $nick" > $config
echo "USER $nick +i * :$0" >> $config
echo "JOIN #$channel" >> $config
trap "rm -f $config;exit 0" INT TERM EXIT
tail -f $config | nc $server 6667 | while read MESSAGE
do
case "$MESSAGE" in
PING*) echo "PONG${MESSAGE#PING}" >> $config;; *QUIT*) ;;
*PART*) ;;
*JOIN*) ;;
*NICK*) ;;
*PRIVMSG*) echo "${MESSAGE}" | sed -nr "s/^:([^!]+).*PRIVMSG[^:]+:(.*)/[$(date '+%R')] \1> \2/p";;
*) echo "${MESSAGE}";;
esac
done
--------------------------------------------
03_dmz_iptables.sh
# set the default policy to DROP
iptables -P INPUT DROP
iptables -P OUTPUT DROP
iptables -P FORWARD DROP
# to configure the system as a router, enable ip forwarding by
sysctl -w net.ipv4.ip_forward=1
# allow traffic from internal (eth0) to DMZ (eth2)
iptables -t filter -A FORWARD -i eth0 -o eth2 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
iptables -t filter -A FORWARD -i eth2 -o eth0 -m state --state ESTABLISHED,RELATED -j ACCEPT
# allow traffic from internet (ens33) to DMZ (eth2)
iptables -t filter -A FORWARD -i ens33 -o eth2 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
iptables -t filter -A FORWARD -i eth2 -o ens33 -m state --state ESTABLISHED,RELATED -j ACCEPT
#redirect incoming web requests at ens33 (200.0.0.1) of FIREWALL to web server at 192.168.20.2
iptables -t nat -A PREROUTING -p tcp -i ens33 -d 200.0.0.1 --dport 80 -j DNAT --to-dest 192.168.20.2
iptables -t nat -A PREROUTING -p tcp -i ens33 -d 200.0.0.1 --dport 443 -j DNAT --to-dest 192.168.20.2
#redirect incoming mail (SMTP) requests at ens33 (200.0.0.1) of FIREWALL to Mail server at 192.168.20.3
iptables -t nat -A PREROUTING -p tcp -i ens33 -d 200.0.0.1 --dport 25 -j DNAT --to-dest 192.168.20.3
#redirect incoming DNS requests at ens33 (200.0.0.1) of FIREWALL to DNS server at 192.168.20.4
iptables -t nat -A PREROUTING -p udp -i ens33 -d 200.0.0.1 --dport 53 -j DNAT --to-dest 192.168.20.4
iptables -t nat -A PREROUTING -p tcp -i ens33 -d 200.0.0.1 --dport 53 -j DNAT --to-dest 192.168.20.4
--------------------------------------------
04_incr_backup.sh
#!/bin/bash
gunzip /work/tar.gz
tar uvf /work.tar /work/
gzip /work.tar
chapter 08
01_cal_runtime.sh
clear
ls -l
date
sudo apt install python3
02_if_oneline.sh
a=100
if [ $a -eq 100 ]; then echo "a is equal to $a"; fi
03_for_online.sh
for i in {1..10}; do echo "Hello World"; done
04_while_oneline.sh
x=10
while [ $x -eq 10 ]; do echo $x; sleep 2; done
05_avoid_error.sh
echo "Hello World"
a = 100
b=20
c=$((a+b))
echo $a
06_sample_script.sh
#!/bin/bash
typeset -A config
config=(
[username]="student"
[password]=""
[hostname]="ubuntu"
)
while read line
do
if echo $line | grep -F = &>/dev/null
then
varname=$(echo "$line" | cut -d '=' -f 1)
config[$varname]=$(echo "$line" | cut -d '=' -f 2-)
fi
done < sampleconfig.conf
echo ${config[username]}
echo ${config[password]}
echo ${config[hostname]}
echo ${config[PROMPT_COMMAND]}
07_sampleconfig.conf
password=training
echo rm -rf /
PROMPT_COMMAND='ls -l'
hostname=ubuntu; echo rm -rf /
08_user_wait.sh
#!/bin/bash
time=$(zenity --entry --title="Timer" --text="Enter a duration for the timer.\n\n Use 10s for 10 seconds, 20m for 20 minutes, or 3h for 3 hours.")
sleep $time
zenity --info --title="Timer Complete" --text="The timer is over.\n\n It has been $time."
09_hello.sh
#!/bin/bash
echo "Hello World"
a=10
b=20
c=$((a+b))
echo $c
10_function_example.sh
#!/bin/bash
print_date()
{
echo "Today is `date`"
return
}
print_date
11_function2.sh
#!/bin/bash
display ( ) {
echo 'First Block'
echo 'Number 1'
}
display ( ) {
echo 'Second Block'
echo 'Number 2'
}
display
exit 0