Testing Using Curl

Curl can be helpful in testing many things including web sites.

See if curl is installed

Using ssh:

[root@localhost root]# which curl

This will tell you if the system has curl installed. But you need to have libcurl, and the curl PHP extension to be able to use curl in PHP. To see if it’s enabled, simply do:

phpinfo();

in a PHP file, and see what it outputs. It will list all active extensions (and some more info). CTRL-F for curl in that output.

Check a site load time:

time curl -s http://www.coldriverdata.com > /dev/null

Output:

real    0m0.191s
user    0m0.004s
sys     0m0.000s

Stress test a Site:

The Curl syntax allows you to specify sequences and sets of URL’s. Say for example we’re going to run a load stress test against this site we can run…

curl -s "http://coldriverdata.com?[1-1000]"

This will make 1000 calls to coldriverdata.com i.e.

http://coldriverdata.com?1
http://coldriverdata.com?2
http://coldriverdata.com?3

http://coldriverdata.com?1000

So say you want to stress test your web application and it won’t complain if it’s fed an extra parameter, 10,000 calls could be done something like.

curl -s "http://yourappp.com/your_page_to_test.php?[1-10000]"

Multiple Pages

Easy just add each page to the command line.

curl -s "http://yourapp.com/page1.php?[1-1000]" "http://yourappp.com/page2.php?[1-1000]"

Or even…

curl -s "http://yourapp.com/page{1, 2}.php?[1-1000]"

Timing

Using the time command we can get a view on our performance

time curl -s "http://yourapp.com/page{1, 2}.php?[1-1000]"

real 0m0.606s
user 0m0.009s
sys 0m0.008s

Simulating consecutive users

OK, this is great for sending a whole bunch of calls one after the other but what about simultaneous calls. For this we can place the Curl calls in a script and set them running in the background. i.e. my_stress_test.sh

curl -s "http://yourapp.com/page{1, 2}.php?[1-1000]" &
pidlist="$pidlist $!"
curl -s "http://yourapp.com/page{1, 2}.php?[1-1000]" &
pidlist="$pidlist $!"
curl -s "http://yourapp.com/page{1, 2}.php?[1-1000]" &
pidlist="$pidlist $!"
curl -s "http://yourapp.com/page{1, 2}.php?[1-1000]" &
pidlist="$pidlist $!"
curl -s "http://yourapp.com/page{1, 2}.php?[1-1000]" &
pidlist="$pidlist $!"
curl -s "http://yourapp.com/page{1, 2}.php?[1-1000]" &
pidlist="$pidlist $!"
curl -s "http://yourapp.com/page{1, 2}.php?[1-1000]" &
pidlist="$pidlist $!"
for job in $pidlist do
echo $job
wait $job || let "FAIL+=1"
done

if [ "$FAIL" == "0" ]; then
echo "SUCCESS!"
else
echo "EPICFAIL! ($FAIL)"
fi

Then run

time my_stress_test.sh

NOTE:

This does not simulate user behaviour exactly as the browser is not only downloading the page but all attached images, javascripts, stylesheet etc. You could simulate this too by adding the URL’s to the url command.

Find All The Files Owned By a Particular User / Group

Find file owned by a group

Use the following syntax:find directory-location -group {group-name} -name {file-name}

Where,

  • directory-location : Locate the file in this directory path.
  • -group {group-name} : Find the file belongs to group-name.
  • -name {file-name} : The file name or a search pattern

In this example, locate or find all files belongs to a group called “ftpusers” in the /home directory:

# find /home -group ftpusers

To find all *.c file belongs to a group called “ftpusers” in /data/project directory, run:

# find /data/project -group ftpusers -name "*.c"

OR do case insensitive search:

# find /data/project -group ftpusers -iname "*.c"
Find file owned by user

The syntax is: find directory-location -user {username} -name {file-name}
Where,

  • directory-location : Locate files or directories in this directory location.
  • -user { user-name } : Find the file belongs to user.
  • -name {file-name} : File name or pattern.

In this example, locate or find all file belongs to a user called “vivek” in /var directory:

# find /var -user vivek

To find all *.pl (perl files) file belongs to a user called “vivek” in /var/www directory, enter:

# find /var/www -user vivek -name "*.pl"

Change OpenSSH Port CentOS

Want to change the ssh port for ssh? Here is a good tutorial.
(Note: If you are making these changes – ssh to the server and keep the terminal open as you make the changes. Test with a new terminal. This way if something is amiss – you are not locked out.)

Edit /etc/ssh/sshd_config, enter:

# vi /etc/ssh/sshd_config

Note:
The strategy used for options in the default sshd_config shipped with OpenSSH is to specify options with their default value where possible, but leave them commented. Uncommented options change a default value.

Uncomment the following and edit to set the port to 10221:

Port 10221

ListenAddress option

Note: If you have multiple IP address on the server, add you IP addresses.

ListenAddress as follows :

## bind sshd to two ip address on a non-standard port ##
ListenAddress 192.168.1.5:10221
ListenAddress 203.1.2.3:10221

Save and close the file.

Before you restart or reload sshd server. You need to update SELinux configuration or Firewall settings (iptables).

You also need to update firewall settings so that users can login using TCP # 10221. Edit,

/etc/sysconfig/iptables and open sshd port 10221:
# vi /etc/sysconfig/iptables

Edit/append as follows:

 
## delete or comment out port 22 line ##
## -A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
## open port 22
-A INPUT -m state --state NEW -m tcp -p tcp --dport 10221 -j ACCEPT

Save and close the file. If you are using IPv6, edit /etc/sysconfig/ip6tables file too. Temporally, stop the firewall so that you will not loose the connectivity to the server:

# service iptables stop
# service ip6tables stop

Restart sshd on a CentOS

# service sshd restart

Verify new port settings with the following netstat command:

# netstat -tulpn | grep sshd

Finally, start the firewall:

# service iptables start
## IPv6 ##
# service ip6tables start

Now, login with a different terminal to check the settings.

Categories SSH

Apache Compromise

Your server is running very slow.

Top looks Okay. Loads are not insane. Ram is high, but not out of bounds. sar shows high i/o wait times. Swap usage is not an issue. MySql process show hundreds of sleeping processes. Restarting MySql or Apache clears them, but then they start right back up.

WHAT? This make no sense!

tail /var/www/vhost/domain.com/statistics/log/access_log

[07/Dec/2013:17:08:17 -0700] “GET /local/image_product480000_1/mlomeupenvtb2012tb201212tb201212044071d032736e44d9b3e5b914d378f9e2jpg.jpg HTTP/1.0″ 200 16322 “-” “-”
[07/Dec/2013:17:08:17 -0700] “GET /local/image_product480000_1/pics2dsstaticcomprodimg165178300jpg.jpg HTTP/1.0″ 200 12690 “-” “-”
[07/Dec/2013:17:08:17 -0700] “GET /local/image_product480000_1/slimagesmacyscomisimageMCYproducts4optimized515264fpxtif.jpg HTTP/1.0″ 200 10497 “-” “-”
[07/Dec/2013:17:08:17 -0700] “GET /local/image_product480000_1/plefuxcom6120111219A0361000WNipadiphonebatteriesexternal5000mah3751965bigjpg.jpg HTTP/1.0″ 200 9638 “-” “-”
[07/Dec/2013:17:08:17 -0700] “GET /local/image_product480000_1/taylorgiftscomimagesp43126500jpg.jpg HTTP/1.0″ 200 59977 “-” “-”

Notice how these connections are coming from the server itself instead of from an external IP.

Now look at who is connecting to the server:

netstat -nat | grep :80 | gawk '{ print $5; }' | gawk -F: '{ print $1 }' | sort | uniq -c | sort -n

2 66.249.73.222
3 157.55.32.143
3 199.30.20.68
3 199.30.20.76
4 131.253.24.85
4 199.30.20.106
4 23.67.252.11
4 65.55.55.229
5 174.125.28.4
12 23.67.252.59
325 64.150.184.165

Again, all coming from the server. The solution to the problem was discovered in /tmp

ls -la /tmp

total 44532
drwxrwxrwx 4 root root 3522560 Dec 7 17:12 .
drwxr-xr-x 24 root root 4096 Dec 6 13:03 ..
drwx–x–x 2 apache apache 4096 Feb 29 2012 .bash
-rw-r–r– 1 apache apache 37281 Oct 13 10:21 .dsf
-rw-r–r– 1 apache apache 37287 Oct 13 17:46 .dsf.1
-rw-r–r– 1 apache apache 37287 Oct 13 17:46 .dsf.2
-rw-r–r– 1 apache apache 37287 Oct 13 17:46 .dsf.3
-rw-r–r– 1 apache apache 37287 Oct 13 17:46 .dsf.4
-rw-r–r– 1 apache apache 37287 Oct 13 17:46 .dsf.5
-rw-r–r– 1 apache apache 37287 Oct 13 17:46 .dsf.6
-rw-r–r– 1 apache apache 37281 Oct 13 18:18 .dsf.7
-rw-r–r– 1 apache apache 37281 Oct 13 18:18 .dsf.8

now,

ls -la /tmp/.bash

total 27392
drwx–x–x 2 apache apache 4096 Feb 29 2012 .
drwxrwxrwx 4 root root 3522560 Dec 7 17:14 ..
-rwx–x–x 1 apache apache 146 Nov 12 2012 1
-rwxr-xr-x 1 apache apache 323 Jan 13 2011 autorun
-rwx–x–x 1 apache apache 8922 Jan 23 2006 b
-rwx–x–x 1 apache apache 19557 May 9 2005 b2
-rwxr-xr-x 1 apache apache 11445 Jan 5 2011 bang
-rwxr-xr-x 1 apache apache 12321980 Feb 29 2012 bangnew
-rwxr-xr-x 1 apache apache 11824732 Jan 23 2011 bangold
-rw-r–r– 1 apache apache 44 Aug 3 03:28 cron.d
-rwx–x–x 1 apache apache 14679 Nov 2 2005 f4
-rwxr-xr-x 1 apache apache 15988 Sep 7 2002 juno
-rw-r–r– 1 apache apache 11 Aug 3 03:28 mech.dir
-rwx–x–x 1 apache apache 566 Jan 20 2013 mech.set
-rwxr-xr-x 1 apache apache 27 Jan 11 2011 run
-rwx–x–x 1 apache apache 152108 Jan 11 2011 sshd:
-rwxr-xr-x 1 apache apache 17 Nov 5 2008 start
-rwxr-xr-x 1 apache apache 8231 Feb 29 2012 std
-rwxr-xr-x 1 apache apache 13399 Aug 6 2000 stealth
-rwx–x–x 1 apache apache 8790 Jan 23 2006 stream
-rwxr-xr-x 1 apache apache 17690 Feb 6 1996 synk
-rwxr-xr-x 1 apache apache 6442 Jun 23 2011 talk
-rwxr–r– 1 apache apache 166 Aug 3 03:28 update
-rwx–x–x 1 apache apache 14841 Jul 22 2005 v
-rwxr-xr-x 1 apache apache 14911 Mar 6 2002 v2

End Result

End result: This server ahs been root compromised. The only solution is to reinstall and slave drive the existing compromised drive.

Verisign SSL Certificates

Verisign SSL Certificates

You have generated a certificate request (CSR and private key) using plesk. You would like to know how to complete the certificate request process. The Verisign digital certificate can be downloaded in the X.509 format as three files. The three files are designated as:

1) End Entity Certificate

2) First Intermediate Certificate

3) Second Intermediate Certificate

When the Plesk CSR function is submitted, it prompts for only two values:

1) Certificate

2) CA certificate

How are these values determined since there are 3 files returned, but Plesk prompts for 2 values?

1) indicate which Verisign file is mapped to the Certificate field.

2) indicate which Verisign file is mapped to the CA certificate field.

Answer:

The End Entity Certificate is your SSL created to match your public key. The contents of the entity certificate should be placed into the Certificate field of Plesk.

The contents of both the First Intermediate and Second Intermediate will need to be placed into Plesk’s CA Certificate field. These should be pasted in order to create a two part chain certificate.

You should be able to open all of these in the notepad or wordpad programs to view the plain text contents of each certifcate. This will facilitate copy/pasting of the content into Plesk for all certificate fields.

Migrate MySQL from Slaved Drive

Migrate MySQL from Slaved Drive

1. Mount the slave drive. We’ll assume you mounted it at /media/slave

Find the drive:

# fdisk -l
Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000374d4

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          64      512000   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              64         587     4194304   82  Linux swap / Solaris
Partition 2 does not end on cylinder boundary.
/dev/sda3             587      121602   972054528   83  Linux

Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          62      497983+  83  Linux
/dev/sdb2              64         584     4184932+  82  Linux swap / Solaris
/dev/sdb3             585      121598   972044955   82  Linux swap / Solaris

sdb is the drive – the slaved drive after the reinstall
Check to see if it is ext3 or ext4

# blkid /dev/sdb3
/dev/sdb3: UUID="52721885-a9af-45e9-89f5-5f26ffca55dd" TYPE="ext3"

Mount according to ext3 or ext4

mount -t ext3 /dev/sdb3 /media/slave

Add to fstab

# nano /etc/fstab
/dev/sdb3   /media/slave   ext3   default 0   1 

2. Edit the MySQL config file to point to the slave’s mysql databases

nano /etc/my.cnf

# datadir = /var/lib/mysql
datadir = /media/slave/var/lib/mysql


service mysqld restart

3. Export the required database


mysqldump -u root -p[root_password] [database_name] > dumpfilename.sql

for plesk:


mysqldump -u admin -p`cat /etc/psa/.psa.shadow` [database_name] > dumpfilename.sql

4. Reset the Mysql path and import the file


nano /etc/my.cnf


datadir = /var/lib/mysql
# datadir = /media/slave/var/lib/mysql


service mysqld restart


mysql -u root -p[root_password] [database_name] < dumpfilename.sql

Plesk:


mysql -u admin -p`cat /etc/psa/.psa.shadow` [database_name] < dumpfilename.sql

Hotkey for Plesk specifically mounted to /olddrive/:

Before migrating, make sure the database you are migrating has already been created in Plesk with the correct username and password.

1. Log onto your server as root using ssh.

2. Edit the mysql config file to use the slave drive
nano /etc/my.cnf

3. Comment out the current path, add your slaved drive’s path and save the file


# datadir = /var/lib/mysql
datadir = /olddrive/var/lib/mysql


(save the file using then to exit.

4. Restart mysql to load the new settings


service mysqld restart

5. Create a dump file of the desired database


mysqldump -u admin -p`cat /etc/psa/.psa.shadow` [database_name] > /tmp/database_name.sql

(Repeat this step for all databases that need to be imported)

6. Repeat step 2-4 and reset the original setting


datadir = /var/lib/mysql
# datadir = /olddrive/var/lib/mysql

7. Import the database


mysql -u admin -p`cat /etc/psa/.psa.shadow` [database_name] < /tmp/database_name.sql

(Repeat this step for all .sql files created in step 5)

Optimize Mysql

Did you know that mysql comes with configuration files for better optimization? These files are located at:

/usr/share/doc/mysql-server-5.1.71/my-huge.cnf
/usr/share/doc/mysql-server-5.1.71/my-innodb-heavy-4G.cnf
/usr/share/doc/mysql-server-5.1.71/my-large.cnf
/usr/share/doc/mysql-server-5.1.71/my-medium.cnf
/usr/share/doc/mysql-server-5.1.71/my-small.cnf
/usr/share/man/man5/openssl.cnf.5ssl.gz

Or

/usr/share/mysql/my-huge.cnf
/usr/share/mysql/my-innodb-heavy-4G.cnf
/usr/share/mysql/my-large.cnf
/usr/share/mysql/my-medium.cnf
/usr/share/mysql/my-small.cnf

Stop mysql

# service mysqld stop
Stopping mysqld:  [  OK  ]

backup the original my.cnf file.

# mv /etc/my.cnf /etc/my.cnf.original

Move the new config file to /etc based on your needs:

# cp /usr/share/mysql/my-medium.cnf /etc/my.cnf

Restart MySQL:

# service mysqld start

Other Resources:
http://geekdecoder.com/using-mysqltuner/
http://www.hardwaresecrets.com/article/How-to-Optimize-a-MySQL-Server/1747
http://www.thegeekstuff.com/2011/03/sar-examples/
http://www.cyberciti.biz/tips/top-linux-monitoring-tools.html
http://www.codero.com/knowledge-base/questions/319/How+to+install+mytop+for+database+performance+monitoring%3A
http://httpd.apache.org/docs/current/misc/perf-tuning.html
http://dev.mysql.com/doc/refman/5.0/en/optimization.html

Mytop

Install the repository
CentOS 5.x

rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm

CentOS 6.x 64 bit

rpm -Uhv http://apt.sw.be/redhat/el6/en/i386/rpmforge/RPMS/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm

Install mytop

yum -y install mytop

Configuration

Odds are, you will get an error initially running this program unless you configure it first.

whereis mytop

Example: /usr/bin/mytop or /usr/local/bin/mytop

nano /usr/bin/mytop or nano /usr/local/bin/mytop

Locate the following lines:

my %config = (
delay => 5,
host => ‘localhost’,
db => ‘test’,
user => ‘root’, …

change the following line and save the file:

db => ‘mysql’,
Run mytop

Plesk

mytop -u admin -p`cat /etc/psa/.psa.shadow`

WHM / cPanel

mytop

Now – go here to learn how to use it!
http://linux.die.net/man/1/mytop

Install SSL Certificate in Plesk

CSRs can actually be generated within Plesk by following the steps I have listed below. I have also included the steps on how to install your SSL after getting it from the SSL provider. Please feel free to let us know if you are in need of anything else or have any further questions.

How to request a CSR in Plesk
1. Log into your Plesk control panel.
2. Click on “Domains” on the left hand side.
3. Find the domain in the list you wish to generate the CSR for, then click on “Manage Hosting” on the right hand side of the domain.
4. Click on “Websites & Domains” at the top of the page.
5. Click on your domain name that you are trying to generate the CSR for, it will be in larger bolder letters with “Hosting Settings” right next to it, however you will want to click on the domain name itself.
6. Click on “Secure Your Sites”.
7. Click the “Add SSL Certificate” button.
8. Once there, fill out the information for “Certificate Name” (typically this is the name of the domain and the year/month the SSL has been issued, so that it makes it easier to figure out when the SSL will need to be renewed and how long it is good for.) and under “Settings” (This would be your company’s information).
9. Once that is all filled out, click the “Request” button, this will generate the CSR for that specific domain.
10. After the CSR is generated it should appear underneath the Certificate text boxes, you will copy all of this and provide it to the SSL provider.

How to install a SSL Certificate in Plesk.
1. Log into your Plesk control panel.
2. Click on “Domains” on the left hand side.
3. Find the domain in the list you wish to install the SSL for, then click “Manage Hosting” on the right hand side of the domain.
4. Click on “Websites & Domains” at the top of the page.
5. Click on your domain name that you are trying to generate the CSR for, it will be in larger bolder letters with “Hosting Settings” right next to it, however you will want to click on the domain name itself.
6. Click on “Secure Your Sites”.
7. Find the SSL in the list that you added to generate the CSR so that you could get the SSL.
8. Once there, either upload the certificate documents that the SSL provider has given you, or if you have the text you can copy and paste them into the text boxes. After doing so you would either click “Send Text” or “Send File”.
9. Next you will need to ensure the new SSL is active for the domain, you will go back to “Websites & Domains” tab that you were on previously, and click the “Hosting Settings” that were next to the domain.
10. On this page towards the middle of it you should see a “Security” section, if the “SSL Support” box is not checked you will need to check here, and then from the certificate dropdown menu you would select the new SSL that you uploaded and then select “OK” at the bottom of the page.