Improving IO performance on Linux

I purchased a new server and have been struggling with IO performance on a RAID 1 setup. I first tried RAID 5 but it was horrible. Through all my struggles, I found a few kernel and filesystem tunables that helped me out.

The virtual machines improved also. They were barely usable.

Filesystem Tweaks: /etc/fstab

Change your mount options to:

defaults,noatime,data=writeback,barrier=0,nobh
Linux Kernel tunables:
sysctl -w vm.swappiness=0
sysctl -w vm.dirty_background_ratio=5
sysctl -w vm.dirty_ratio=5
sysctl -w vm.vfs_cache_pressure=200
The dirty tunables controls the portion of memory that the kernel will store pages in. I decreased the value to 5 so that I will have more memory free.  The cache_pressure tunable tells the kernel how to quickly free up the cache.

How to clone a KVM virtual machine on Linux

First, select the virtual machine that you want to clone.

 

[root@redhat ~]# ls /etc/libvirt/qemu/*.xml

/etc/libvirt/qemu/cp1.xml     /etc/libvirt/qemu/vcs5_sol_n1.xml      /etc/libvirt/qemu/vcs6_lin_vvr_n3.xml

/etc/libvirt/qemu/rhel6.xml   /etc/libvirt/qemu/vcs6_lin_vvr_n1.xml  /etc/libvirt/qemu/vcs6_sol_n1.xml

/etc/libvirt/qemu/rusher.xml  /etc/libvirt/qemu/vcs6_lin_vvr_n2.xml

Continue reading How to clone a KVM virtual machine on Linux

Removing logging to increase I/O performance on the Raspberry Pi

I recently was looking at optimizing the device to make Apache run faster. Thanks to TAFB from #raspberrypi on FreeNode IRC chat, I experimented with turning off syslog and noticed a few second increase with the two different sites hosted on the Pi. Overall, there will be less IO on the system and less RAM and CPU being used that will result in slightly better performance on the PI. Also, there will be significantly less writes on the SD card. SD cards have a limited number of writes so it is important to reduce the numbers to have a longer lifespan.

The distribution used is Arch Linux.

In /etc/systemd/journald.conf:
Storage=none

Manage The Services:
# systemctl stop systemd-journald
# systemctl disable systemd-journald

Remove Syslog:
# systemctl stop syslog-ng.service
# systemctl disable syslog-ng.service
# pacman -R syslog-ng

Next, I began to tune root filesystem for optimal performance and disabled logging.

Add the following to /etc/fstab:

/dev/mmcblk0p2 / ext4 rw,noatime,data=writeback,barrier=0,nobh,errors=remount-ro 0 1

# reboot

# tune2fs -o journal_data_writeback /dev/mmcblk0p2

Before: Rusher81572.com loaded in 2.857s
After: Rusher81572.com loaded in 2.149s

Before: Rusher81572.com/pi loaded in 4.356s
After: Rusher81572.com/pi loaded in 4.039s

Dovecot/Postfix Email Server setup with Gmail Relay

I have been struggling to get an email server setup and working by following many guides online. Here is how I got it working. I had to use Gmail as an relayhost to actually send the email from the server because my ISP blocks me from sending mail on port 25 and doesn’t allow me to relay mail through their servers because I’m using me@mydomain.com instead of me@isp.com)

Software: postfix and dovecot
Operating System: CentOS and Archlinux
(CentOS users need to install cyrus-sasl-plain also )

SASL Configuration

Modify or create: /usr/lib/sasl2/smtpd.conf

pwcheck_method:saslauthd
saslauthd_path: /var/run/saslauthd/mux
SASLAUTHD_OPTS=”${SASLAUTH_MECH} -a pam”
mech_list: plain login
log_level: 7

Dovecot Configuration

Copy the dovecot configuration files :

# cp /usr/share/doc/dovecot/example-config/* /etc/dovecot/

Modify /etc/dovecot/dovecot.conf:

protocols = imap

Modify: /etc/dovecot/conf.d/10-auth.conf :

disable_plaintext_auth = no

Modify: /etc/dovecot/conf.d/10-mail.conf :

mail_location = maildir:~/Maildir

Create or Modify: /etc/pam.d/dovecot

auth required pam_unix.so nullok
account required pam_unix.so

Postfix Configuration

Modify /etc/postfix/main.cf:

myhostname = rusher81572.com
mydomain = rusher81572.com
myorigin = $mydomain
inet_interfaces = all
mydestination = $myhostname, localhost.$mydomain, localhost
relayhost = [smtp.gmail.com]:587
home_mailbox = Maildir/
smtp_use_tls=yes
smtp_sasl_auth_enable = yes

smtpd_sasl_auth_enable = yes

smtp_sasl_password_maps = hash:/etc/postfix/password

smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_sasl_security_options =
smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks,check_relay_domains

Create /etc/postfix/password:

[smtp.gmail.com]:587 username:password

#postmap /etc/postfix/password

#systemctl start postfix

Create a DHCP and DNS server with RedHat/ CentOS Linux

#yum -y install bind bind-utils dhcp

 

Tell what interface to use with dhcpd:   /etc/sysconfig/dhcpd

DHCPDARGS=eth2

 

In /etc/named.conf add: (Modify to suit the following)

allow-query     { any; };

allow-recursion {any; };

listen-on port 53 { 10.10.10.2; };

 

Add this zone near the bottom:

 

zone “dhcpserver” IN {

type master;

file “dhcpserver”;

allow-update { any; };

notify no;

};

 

Create /var/named/dhcpserver:

 

@       IN      SOA     dhcpserver. hostmaster.dhcpserver. (

2012080701      ; Serial

43200      ; Refresh

3600       ; Retry

3600000    ; Expire

2592000 )  ; Minimum

;       Define the nameservers and the mail servers

IN      NS      dhcpserver.

IN      A       10.10.10.2

ns            IN      A       10.10.10.2

www        IN      A       10.10.10.2

client        IN      A       10.10.10.3

gateway        IN      A       10.10.10.2

 

Only have this in /etc/dhcp/dhcpd.conf:

 

subnet 10.10.10.0 netmask 255.255.255.0 {

option routers                  10.10.10.2;

option subnet-mask              255.255.255.0;

option domain-name              “dhcpserver”;

option domain-name-servers       10.10.10.2;

option time-offset              -18000;     # Eastern Standard Time

range 10.10.10.20 10.10.10.30;

}

 

#chkconfig named on

#chkconfig dhcpd on

#service named start

#service dhcpd start

Convert root partition to LVM and mirror the root disk

1. Create a tar backup of your filesystem.

# tar czpf /root/redhat.tar –exclude=/var/tmp/portage/* –exclude=/root/* –exclude=/usr/portage/* –exclude=*.deb –exclude=/tmp/* –exclude=*.rpm –exclude=/sys/* –exclude=/proc/* –exclude=/dev/* –exclude=/mnt/* –exclude=/media/* –exclude=/home/* –exclude=/var/lib/libvirt/images/* –exclude=/oracle/* –exclude=redhat.tar

2. Use fdisk to create /boot and 1 LVM partition on the new disk.

/dev/sda1 * 1 100 803218+ 83 Linux
/dev/sda2 101121601 975956782+ 8e Linux LVM

3. Set /dev/sda1 to be bootable.

# parted /dev/sda set 1 boot on

4. Create the new LVM partition.

# pvcreate /dev/sda2
# vgcreate vg /dev/sda2
# lvcreate -L 200G /dev/vg -n root
# mkfs /dev/vg/root
# mkfs /dev/sda1
# mount /dev/vg/root /mnt
# mount /dev/sda1 /mnt/boot

5. Extract the tar file to /mnt

# tar xpf /root/redhat.tar -C /mnt/

6. Modify the following files:

/mnt/boot/grub/menu.list

Modify the kernel line to support LVM by adding the following LVM details:

rd_LVM_VG=vg rd_LVM_LV=root

Also ensure that initrd and kernel does not have /boot/ in the location.

Example:

kernel /vmlinuz-2.6.32-279.2.1.el6.x86_64 ro root=/dev/mapper/vg-root LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=128M rd_LVM_VG=vg rd_LVM_LV=root rhgb quiet

initrd /initramfs-2.6.32-279.2.1.el6.x86_64.img

/mnt/etc/fstab:

Change the /boot and / entries to LVM:

/dev/sda1 /boot ext4 defaults 0 0
/dev/mapper/vg-root / ext4 defaults 1 1

7. Mount and configure the new environment:

# mount /dev/vg/root /mnt
# mount /dev/sda1 /mnt/boot
# mount -o bind /sys /mnt/sys
# mount -o bind /dev /mnt/dev
# mount -o bind /proc /mnt/proc
# grep -v rootfs /proc/mounts > /mnt/etc/mtab

Modify /mnt/etc/mtab and add:

/dev/sda1 /boot ext4 rw 0 0

# chroot /mnt

8. Install GRUB and reconfigure the ram disk image:

# grub-install –recheck /dev/sda
# dracut -force

9. Unmount and reboot:

Type exit to exit the chroot environment
# cd /
# umount /mnt/*
# umount /mnt
# reboot

Set your system to boot from the disk known as /dev/sda

10. Initialize and format your original boot disk.

Just like we did for /dev/sda. (1 bootable partition for /boot and 1 Linux LVM partition.

Device Boot Start End Blocks Id System
/dev/sdc1 * 1 100 803218+ 83 Linux
/dev/sdc2 101 121601 975956782+ 8e Linux LVM

11. Add /dev/sdc to the volume group.
# vgextend /dev/vg /dev/sdc2

12. Format the boot partition on the drive and set it bootable:
# mkfs /dev/sdc1
# parted /dev/sdc set 1 boot on

13. Mirror the boot disk:

# lvconvert -m1 /dev/vg/root