Wednesday, October 31, 2012

Friday, October 26, 2012

Time of command execution

How to get time of execution of any command in Unix/Linux

Simply execute the following command on shell prompt and you will get date and time execution of any command.

$ export HISTTIMEFORMAT='%F %T '

To make it permanent edit the

To display timezone:

$ export HISTTIMEFORMAT='%d-%b-%Y %r %Z'

$ history

351 08-Sep-2009 09:53:47 PM PDT ls -lrt

352 08-Sep-2009 09:53:49 PM PDT cd ..

To display AM / PM: You can use %r option

To display in dd-MON-YY time format: You can format it as you want %d-%b-%Y %r

Tuesday, October 23, 2012

Find examples with exec command

Search files with find and delete them with exec, this is probably one of the most common actions with exec, and you should not use exec for this, read later, here are some examples of common uses:

1) List all files starts with “abc”:

 find / -name "abc*" -exec /bin/ls {} \;

2) Search all files start with “abc” and delete them:

 find / -name "abc*" -exec /bin/rm {} \;

3) Search all files with size > of 10 MB and delete them:

 find / -size +10M -exec /bin/rm {} \;

Sometimes some programs goes wild and create thousands of small files into one directoy, in this case you cannot use a simple

rm
*

because the shell would not be able to manages the expansion of the character * with all these file names, but you can use find to delete all files in a directory one by one.

 find . -exec /bin/rm {} \;

You should NOT use these examples, In newer verison you will find the option -delete which is safer then “-exec /bin/rm {} ;”. For example:

find / -name "*.old" -delete

In older Unix system you could not have the -delete option, and so you have no choice but to use the -exec action.

4) To change permissions on files recursively, leave directories alone.

find ./ -type f -exec chmod 755 {} \;

5) With the option -type f you select only the files and after that is easy to do a chmod on them. Recursively change the ownership of all the files from old user to new user

find / -user test_old  -type f  -exec chown  test_new {} \;

6) Recursively change the permissions of all, and only, the directory

find . -type d -exec chmod 655 {} \;

In this example I’ve used again the option -type with d parameter to identify only the directories.

Monday, October 22, 2012

Shell Script Interview questions and answers

Basic shell scripting questions

How do you find out what’s your shell? - echo $SHELL
What’s the command to find out today’s date? - date
What’s the command to find out users on the system? - who
How do you find out the current directory you’re in? - pwd
How do you remove a file? - rm
How do you remove a - rm -rf
How do you find out your own username? - whoami
How do you send a mail message to somebody? - mail somebody@techinterviews.com -s ‘Your subject’ -c ‘cc@techinterviews.com‘
How do you count words, lines and characters in a file? - wc
How do you search for a string inside a given file? - grep string filename
How do you search for a string inside a directory? - grep string *
How do you search for a string in a directory with the subdirectories recursed? - grep -r string *
What are PIDs? - They are process IDs given to processes. A PID can vary from 0 to 65535.
How do you list currently running process? - ps
How do you stop a process? - kill pid
How do you find out about all running processes? - ps -ag
How do you stop all the processes, except the shell window? - kill 0
How do you fire a process in the background? - ./process-name &
How do you refer to the arguments passed to a shell script? - $1, $2 and so on. $0 is your script name.
What’s the conditional statement in shell scripting? - if {condition} then … fi
How do you do number comparison in shell scripts? - -eq, -ne, -lt, -le, -gt, -ge
How do you test for file properties in shell scripts? - -s filename tells you if the file is not empty, -f filename tells you whether the argument is a file, and not a directory, -d filename tests if the argument is a directory, and not a file, -w filename tests for writeability, -r filename tests for readability, -x filename tests for executability
How do you do Boolean logic operators in shell scripting? - ! tests for logical not, -a tests for logical and, and -o tests for logical or.
How do you find out the number of arguments passed to the shell script? - $#
What’s a way to do multilevel if-else’s in shell scripting? - if {condition} then {statement} elif {condition} {statement} fi
How do you write a for loop in shell? - for {variable name} in {list} do {statement} done
How do you write a while loop in shell? - while {condition} do {statement} done
How does a case statement look in shell scripts? - case {variable} in {possible-value-1}) {statement};; {possible-value-2}) {statement};; esac
How do you read keyboard input in shell scripts? - read {variable-name}
How do you define a function in a shell script? - function-name() { #some code here return }
How does getopts command work? - The parameters to your script can be passed as -n 15 -x 20. Inside the script, you can iterate through the getopts array as while getopts n:x option, and the variable $option contains the value of the entered option.

Shell scripts

Check multiple websites' status.

[ Note: First copy the check_http plugin in /usr/lib/nagios/plugins/ path ]

#!/bin/bash

#####

#Script will check all given websites and give status by the help of check_http plugin

#Created by Shankar Patel

#Create date :10-10-2012

#######################################################################

F_P=""
msg_ok=""
exit_status=0

#also you can create a file with the list of websites and

#SITES_LIST="file_name_of_list_websites"

#for WEB in $(cat $SITES_LIST)
for WEB in {staging.sabsebolo.com,staging.sabsetalk.com,google.com,yyyyyyyyyyyyyyyyahoo.com}
do
/usr/lib/nagios/plugins/check_http $WEB > /dev/null
result_check=$?
if [ "$result_check" -ne 0 ] ; then
       F_P="$F_P $WEB"
       exit_status=2
else
       msg_ok="$msg_ok $WEB"
       exit_status=0
fi
done
if [ "$F_P" = "" ]
then
      echo "OK. All Websites are OK."
else
      echo "CRITICAL. $F_P websites showing down. and $msg_ok are up."
      exit $exit_status;
fi

Script Daily Backup To remote location

#!/bin/bash

SSHKEY=/backup/id_rsa

TOBACKUP=$1

HOSTNAME=$(hostname)

TODAY=`date +%Y%m%d`

check_time=0

tar_done=1

#create tar file

tak_bkp()

{

if [ "$check_bkp" -lt 4 ]

then

scp -o StrictHostKeyChecking=no -i $SSHKEY /backup/${HOSTNAME}_${TODAY}.tar.gz incoming-backup@heimdall:/mnt/datastore1/incoming-backup/ > /backup/backup_done

[ "$?" -eq 0 ] && rm /backup/${HOSTNAME}_${TODAY}.tar.gz && exit 0 || check_bkp=$(($check_bkp+1)) && tak_bkp ;

else

echo -e "Can not create BKP file! \n ackup not done properly" > /backup/backup_done

exit 2;

}

create_tar()

{

if [ "$check_tar" -lt 4 ]

then

tar -cpzf /backup/${HOSTNAME}_${TODAY}.tar.gz --exclude="${HOSTNAME}_${TODAY}.tar.gz" --one-file-system $TOBACKUP

[ "$?" -eq 0 ] && tak_bkp

check_time=$(($check_time+1))

else

echo -e "Can not create tar file! \nbackup not done properly" > /backup/backup_done

exit 2;

}

create_tar

Script to check backup done or not

#!/bin/bash

#####

#Script will check backup copied or not on remote location.

#Created by Shankar Patel

#Create date :10-10-2012

#######################################################################

TODAY=`date +%Y%m%d --date="1 day ago"`

bkp_exist=`ssh -o StrictHostKeyChecking=no -i /backup/id_rsa incoming-backup@heimdall "ls /mnt/datastore1/incoming-backup/${HOSTNAME}_${TODAY}.tar.gz &>/dev/null && echo -e 'OK' || echo -e 'CRITICAL'" 2>/dev/null`

[ "$bkp_exist" == "OK" ] && echo "OK. Yesterday's backup done." && exit 0

[ "$bkp_exist" == "CRITICAL" ] && echo "CRITICAL. Yesterday's backup done." && exit 2

Nagios plugin to check Openfiles

check_openfiles.sh

if it is greater then 85% then it will show warning

if it is greater then 90% then it will show warning

otherwise it will show ok.

#!/bin/bash

###########

# This script will check no open files and if open files will be greater then 85% of max-open-files then it will show warning and open files will be greater then 90% then it will show critical.

# It does not require any additional perameter at execution time

# it automatically takes perameter from system files

########### Shankar Patel 22/08/1012

open_files=`cat /proc/sys/fs/file-nr | awk '{print $1}'`

max_open_files=`cat /proc/sys/fs/file-nr | awk '{print $3}'`

warn_of=$(($max_open_files*15/100))

crit_of=$(($max_open_files*10/100))

if [ "$open_files" -lt "$warn_of" ]

then

echo "OK. Number of open files = $open_files | openfiles= $open_files"

exit 0;

elif [ "$open_files" -gt "$warn_of" ] && [ "$open_files" -lt "$crit_of" ]

then

echo "WARNING. Number of open files = $open_files | openfiles= $open_files"

exit 1;

else

echo "CRITICAL. Number of open files = $open_files | openfiles= $open_files"

exit 2;

Script to monitor log file error.

#!/bin/bash

FILE2TAIL=$1

PATTERN=$2

CATCH_UP=0

FPATH=$(dirname $1)

FFILE=$(basename $1)

INODE_NUM=$(ls -i ${FPATH}/${FFILE} | awk '{ print $1 }')

PATTERN_CODE="$(echo -n "$PATTERN"|tr -d \* | tr -d ' ' | tr -d \. )"

FSIZE_FILE=${FPATH}/.${FFILE}_${PATTERN_CODE}_${INODE_NUM}.size

SIZE_CURRENT=$(ls -l ${FPATH}/${FFILE} | awk '{ print $5 }')

exit_status=0

last_exit()

{

# save current size

ls -l ${FPATH}/${FFILE} | awk '{ print $5 }' > ${FSIZE_FILE}

exit $exit_status

}

if [ ! -f ${FSIZE_FILE} ]; then

# first time, save line count

echo "saving initial size of ${FPATH}/${FFILE}"

last_exit

# last size is available

SIZE_LAST=$(cat ${FSIZE_FILE})

SIZE_DELTA=$(($SIZE_CURRENT - $SIZE_LAST))

if [ $SIZE_DELTA -gt 0 ]

then

# new bytes to scan

tail "-${SIZE_DELTA}c" ${FPATH}/${FFILE} > /tmp/ts$$

egrep "$PATTERN" /tmp/ts$$ > /tmp/tsF$$

GREP_STATUS=$?

if [ $GREP_STATUS -eq 0 ]

then

# found a match

SUBJECT="Found $PATTERN in $FILE2TAIL"

BODY="$SUBJECT\n$(cat /tmp/tsF$$)"

echo "$SUBJECT"

#### exec

#cho -e $BODY # | $EXEC_ON_MATCH -s "$SUBJECT" $EMAILADDR

echo "CRITICAL. 'Error writing data to the connection' found in ${FFILE} | error_found=1"

last_exit

if [ $GREP_STATUS -ne 0 ]

then

# found no matches

SUBJECT="Missing $PATTERN in $FILE2TAIL on $HOST_NAME"

BODY="$SUBJECT\n"

echo "OK. 'Error writing data to the connection' not found in ${FFILE} | error_found=0"

last_exit

#### exec

/bin/rm -f /tmp/ts$$ /tmp/tsF$$

find out notusable data in linux

How to keep disk-space to free from not usable data in LINUX/Unix:

Agedu: Correlate disk usage with last-access times to identify large and disused data
agedu scans a directory tree and produces reports about how much disk space is used in each directory and subdirectory, and also how that usage of disk space corresponds to files with last-access times a long time ago.
In other words, agedu is a tool you might use to help you free up disk space. It lets you see which directories are taking up the most space, as du does; but unlike du, it also distinguishes between large collections of data which are still in use and ones which have not been accessed in months or years - for instance, large archives downloaded,unpacked, used once, and never cleaned up. Where du helps you find what's using your disk space, agedu helps you find what's wasting your disk space.

How to install:
==> for Debian / Ubuntu Linux user type the following apt-get command to install agedu:
$sudo apt-get install agedu

==> for RHEL / CentOS / Fedora users turn on EPEL repo and type the following
#yum install agedu

Example how to use it:

Step1: The following command will conllect information about disk used by files in /home/shankar directory.

agedu -s /home/shankar/

Step2: Then execute the following command on command prompt to view the disk usage in web browser:

agedu -w

you will see the following output :

Using Linux /proc/net magic authentication

URL: http://127.0.0.1:42823/

Use this url in web browser and you will see the file and folder level usage graphically.

Note: Don't stop this command.

For more information use man agedu

Thursday, October 18, 2012

ss command

How to monitor current TCP and UDP connections

ss :- Used to investigate tcp/udp sockets

USAGE EXAMPLES

ss -t -a

Display all TCP sockets.

ss -u -a

Display all UDP sockets.

ss -o state established '( dport = :ssh or sport = :ssh )'

Display all established ssh connections.

ss -x src /tmp/.X11-unix/*

Find all local processes connected to X server.

ss -o state fin-wait-1 '( sport = :http or sport = :https )' dst 193.233.7/24

List all the tcp sockets in state FIN-WAIT-1 for our apache to network 193.233.7/24 and look at their timers.

MAN page :

DESCRIPTION

ss is used to dump socket statistics. It allows showing information similar to netstat. It can display more TCP and state informations than other tools.

OPTIONS

These programs follow the usual GNU command line syntax, with long options starting with two dashes (`-'). A summary of options is included below.

-h, --help

Show summary of options.

-V, --version

Output version information.

-n, --numeric

Do now try to resolve service names.

-r, --resolve

Try to resolve numeric address/ports.

-a, --all

Display all sockets.

-l, --listening

Display listening sockets.

-o, --options

Show timer information.

-e, --extended

Show detailed socket information

-m, --memory

Show socket memory usage.

-p, --processes

Show process using socket.

-i, --info

Show internal TCP information.

-s, --summary

Print summary statistics. This option does not parse socket lists obtaining summary from various sources. It is useful when amount of sockets is so huge that parsing /proc/net/tcp is painful.

-4, --ipv4

Display only IP version 4 sockets (alias for -f inet).

-6, --ipv6

Display only IP version 6 sockets (alias for -f inet6).

-0, --packet

Display PACKET sockets.

-t, --tcp

Display only TCP sockets.

-u, --udp

Display only UDP sockets.

-d, --dccp

Display only DCCP sockets.

-w, --raw

Display only RAW sockets.

-x, --unix

Display only Unix domain sockets.

-f FAMILY, --family=FAMILY

Display sockets of type FAMILY. Currently the following families are supported: unix, inet, inet6, link, netlink.

-A QUERY, --query=QUERY

List of socket tables to dump, separated by commas. The following identifiers are understood: all, inet, tcp, udp, raw, unix, packet, netlink, unix_dgram, unix_stream, packet_raw, packet_dgram.

-D FILE

Do not display anything, just dump raw information about TCP sockets to FILE after applying filters. If FILE is - stdout is Manual page ss(8) line 47

-f FAMILY, --family=FAMILY

Display sockets of type FAMILY. Currently the following families are supported: unix, inet, inet6, link, netlink.

-A QUERY, --query=QUERY

List of socket tables to dump, separated by commas. The following identifiers are understood: all, inet, tcp, udp, raw, unix,

packet, netlink, unix_dgram, unix_stream, packet_raw, packet_dgram.

-D FILE

Do not display anything, just dump raw information about TCP sockets to FILE after applying filters. If FILE is - stdout is

used.

-F FILE, --filter=FILE

Read filter information from FILE. Each line of FILE is interpreted like single command line option. If FILE is - stdin is

used.

FILTER := [ state TCP-STATE ] [ EXPRESSION ]

Please take a look at the official documentation (Debian package iproute-doc) for details regarding filters.

PS command

ps ax -o user,vsz,rss,pcpu,command --columns 10000 | sed -e 1d -e 's/ *$[^ ]*$ *$[^ ]*$ *$[^ ]*$ *$[^ ]*$ */(\1,\2,\3,\4) /'

Wednesday, October 17, 2012

Secure web server configuration

Step 1: Generate a Private Key

The openssl toolkit is used to generate an RSA Private Key and CSR (Certificate Signing Request). It can also be used to generate self-signed certificates which can be used for testing purposes or internal usage.
The first step is to create your RSA Private Key. This key is a 1024 bit RSA key which is encrypted using Triple-DES and stored in a PEM format so that it is readable as ASCII text.
openssl genrsa -des3 -out server.key 1024Generating RSA private key, 1024 bit long modulus
.........................................................++++++
........++++++
e is 65537 (0x10001)
Enter PEM pass phrase:
Verifying password - Enter PEM pass phrase:

Step 2: Generate a CSR (Certificate Signing Request)

Once the private key is generated a Certificate Signing Request can be generated. The CSR is then used in one of two ways. Ideally, the CSR will be sent to a Certificate Authority, such as Thawte or Verisign who will verify the identity of the requestor and issue a signed certificate. The second option is to self-sign the CSR, which will be demonstrated in the next section.

During the generation of the CSR, you will be prompted for several pieces of information. These are the X.509 attributes of the certificate. One of the prompts will be for "Common Name (e.g., YOUR name)". It is important that this field be filled in with the fully qualified domain name of the server to be protected by SSL. If the website to be protected will be https://public.akadia.com, then enter public.akadia.com at this prompt. The command to generate the CSR is as follows:

openssl req -new -key server.key -out server.csr

Country Name (2 letter code) [GB]:CH

State or Province Name (full name) [Berkshire]:Bern

Locality Name (eg, city) [Newbury]:Oberdiessbach

Organization Name (eg, company) [My Company Ltd]:Akadia AG

Organizational Unit Name (eg, section) []:Information Technology

Common Name (eg, your name or your server's hostname) []:public.akadia.com

Email Address []:martin dot zahn at akadia dot ch

Please enter the following 'extra' attributes

to be sent with your certificate request

A challenge password []:

An optional company name []:

Step 3: Remove Passphrase from Key

One unfortunate side-effect of the pass-phrased private key is that Apache will ask for the pass-phrase each time the web server is started. Obviously this is not necessarily convenient as someone will not always be around to type in the pass-phrase, such as after a reboot or crash. mod_ssl includes the ability to use an external program in place of the built-in pass-phrase dialog, however, this is not necessarily the most secure option either. It is possible to remove the Triple-DES encryption from the key, thereby no longer needing to type in a pass-phrase. If the private key is no longer encrypted, it is critical that this file only be readable by the root user! If your system is ever compromised and a third party obtains your unencrypted private key, the corresponding certificate will need to be revoked. With that being said, use the following command to remove the pass-phrase from the key:

cp server.key server.key.org

openssl rsa -in server.key.org -out server.key

The newly created server.key file has no more passphrase in it.

-rw-r--r-- 1 root root 745 Jun 29 12:19 server.csr

-rw-r--r-- 1 root root 891 Jun 29 13:22 server.key

-rw-r--r-- 1 root root 963 Jun 29 13:22 server.key.org

Step 4: Generating a Self-Signed Certificate
At this point you will need to generate a self-signed certificate because you either don't plan on having your certificate signed by a CA, or you wish to test your new SSL implementation while the CA is signing your certificate. This temporary certificate will generate an error in the client browser to the effect that the signing certificate authority is unknown and not trusted.
To generate a temporary certificate which is good for 365 days, issue the following command:
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crtSignature ok
subject=/C=CH/ST=Bern/L=Oberdiessbach/O=Akadia AG/OU=Information
Technology/CN=public.akadia.com/Email=martin dot zahn at akadia dot ch
Getting Private key

Step 5: Installing the Private Key and Certificate

When Apache with mod_ssl is installed, it creates several directories in the Apache config directory. The location of this directory will differ depending on how Apache was compiled.

cp server.crt /usr/local/apache/conf/ssl.crt

cp server.key /usr/local/apache/conf/ssl.key

Step 6: Configuring SSL Enabled Virtual Hosts

SSLEngine on

SSLCertificateFile /usr/local/apache/conf/ssl.crt/server.crt

SSLCertificateKeyFile /usr/local/apache/conf/ssl.key/server.key

SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown

CustomLog logs/ssl_request_log \

"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

Step 7: Restart Apache and Test

/etc/init.d/httpd stop

Scripting and Automation are Skills with which you shine in crowd

Total Hit Counter

Wednesday, October 31, 2012

Linux commands

Friday, October 26, 2012

Time of command execution

Tuesday, October 23, 2012

Find examples with exec command

Monday, October 22, 2012

Shell Script Interview questions and answers

Shell scripts

find out notusable data in linux

How to keep disk-space to free from not usable data in LINUX/Unix:

Thursday, October 18, 2012

ss command

How to monitor current TCP and UDP connections

Wednesday, October 17, 2012

Secure web server configuration