Nombre total de pages vues

mercredi 27 juillet 2011

MySQL-MHA: HA replication tool

A new MySQL project about high availability is proposed by Yoshinori Matsunobu. This project permits to automating master failover and slave promotion within short downtime.

This is the first time that we can see a serious MySQL failover project into replication environment. I haven't tested it yet but lot of people speak about MySQL-MHA on the web actually. Yoshinori Matsunobu is associate with SkySQL for that they provide the commercial support.

MySQL-MHA brings a lot of advantages such as:

  • Automated master monitoring and failover
  • Interactive (manual) Master Failover
  • Non-interactive master failover
  • Online switching master to a different host

Every informations and descriptions can be found on the google code web page of MySQL-MHA.
You can know more about Yoshinori Matsunobu on his blog.

Project is released few days ago and I think that it will no need to wait for a long time to seeing this marvelous tools into your MySQL production replication environment.


lundi 25 juillet 2011

Cobbler installation server

INTRODUCTION
Cobbler is an installation server that permits to install easily and automatically operating systems, virtualized or not with PXE. We will describe installation, configuration and use into this howto.
For this howto we use a CentOS 5 x86_64 distro.


INSTALLATION
First thing to do is to configure DHCP server. In our case, Cobbler server is not on the same machine that DHCP server, so we have to register the PXE class for Cobbler server on DHCP server.
Here is the part of dhcpd.conf config file regarding Cobbler service. After configuration, we have to restart the DHCP service to validate changes:
vi /etc/dhcp/dhcpd.conf

        host cobbler
        {
                option host-name "cobbler";
                ddns-hostname "cobbler";
                hardware ethernet 00:00:00:00:00:00; #MAC address of cobbler server
                fixed-address 192.168.0.XX; #IP of Cobbler server
                allow booting;
                allow bootp;
                class "PXE" {
                match if substring(option vendor-class-identifier, 0, 9) = "PXEClient";
                next-server 192.168.0.XX; #IP of Cobbler server
                filename "pxelinux.0";
        }
        }

:wq

/etc/init.d/dhcpd restart
Next, an easy yum command permits to install the Cobbler service on the server:
yum install PyYAML cobbler cobbler-web
Installation is now finished.


CONFIGURATION
For configuration, we will initially adjust cobbler parameters, we have to define an encrypted password that will be the root password for the machine installed with PXE and we define IP address for cobbler server.
We begin by creating the encrypted password using htpasswd:
/usr/bin/htpasswd -c /home/htpasswd root
New password: [cobbler]
Re-type new password: [cobbler]
Adding password for user root

cat /home/htpasswd

root:xH.KydRoK8CyQ
The password to use in the config file of Cobbler server is located after 'root', now we can edit the config file of Cobbler server:
vi /etc/cobbler/settings

default_password_crypted: xH.KydRoK8CyQ

next_server: 192.168.0.XX #IP of Cobbler server

server: 192.168.0.XX #IP of Cobbler server

:wq
Then we will install the services needed to run cobbler:
yum install tftp tftp-server xinetd
/etc/init.d/xinetd start
/etc/init.d/cobblerd start
chkconfig xinetd on
chkconfig cobblerd on
Now we will monitor the proper Cobbler configuration and adjust the parameters as will be shown:
cobbler check

/etc/init.d/httpd start

vi /etc/xinetd.d/rsync
 disable = no
:wq

yum install cman

cobbler get-loaders

yum install debmirror
vi /etc/debmirror.conf
 # @dists="sid";
 # @arches="i386";
:wq

/etc/init.d/cobblerd restart

cobbler sync
cobbler check
Cobbler is now properly configured. The next step is to import the ISO distribution that we want to make it available via PXE. In our case we will use CentOS and import can not be done with the netinstall ISO and ISO DVD includes too many packages that do not serve us.
We have so customized CentOS using CD1 of CentOS. A Howto was written on this blog about customization of CentOS, you can find it here:  Customize CentOS
We are going to use this custom ISO file for the import into Cobbler server.
First, we must download this ISO file locally (on Cobbler server) then we gonna mount it and import it.
mount -t auto -o loop /downloads/CentOS-5.5-x86_64-core.iso /mnt/
cobbler import --path=/mnt --name=Centos5 –arch=x86_64

cobbler sync
cobbler list

distros:
   Centos5-x86_64
   Centos5-xen-x86_64

profiles:
   Centos5-x86_64
   Centos5-xen-x86_64

systems:

repos:

images:
We can see in this list (distros and profiles) that CentOS ISO was added to Cobbler server. Now we have to create local repository. We could use the official repo CentOS but be aware that the cobbler will grab all the packages locally from the repo.
This operation takes time and a lot of disk space. Knowing that we have no need of all packages available on the official CentOS repo we will create a local repo with little packets which will occupy little disk space but more importantly to make the installation via PXE even faster.
So to create the local repo, we have to install a virtual machine using the custom CentOS ISO created previously (howto on the blog) and then we're gonna download all installed packages locally (on the virtual machine) for create the local repository.
I don't detail the installation of a VM from customized CentOS ISO (I assume you know how to do this).
To download installed packages, we're gonna use a method described previously on this blog too (here is the article: Download RPM packages with yum): with yumdownloader program
mkdir /tmp/rpm
yum update -y
yum install yum-utils -y
rm -rf /root/rpm.log &&rpm -qa > rpm.log
for i in $(cat /root/rpm.log) ; do yumdownloader $i --destdir=/tmp/rpm ; done
Then, we declare this local repo on the Cobbler server. To do this we copy every rpm packages into /tmp/rpm directory's on the VM to /tmp/rpm on the Cobbler server:
# Copy packages from VM to Cobbler server
scp -r /tmp/rpm/* user@IP_COBBLER:/tmp/rpm/
# Declare Cobbler local repo
cobbler repo add --mirror=/tmp/rpm/ --name=centos5-local
cobbler reposync
We can see now the cobbler local repo:
cobbler list

repos:
   centos5-local
Then we will edit the default kickstart file, this file permits to automate CentOS install:
cd /var/lib/cobbler/kickstarts/
mv sample.ks sample.ks_bak
vi sample.ks

auth --useshadow --enablemd5
bootloader --location=mbr
clearpart --all --initlabel
text
firewall --disabled
firstboot --disable
keyboard fr-pc
lang fr_FR.UTF-8
url --url=$tree
$yum_repo_stanza
$SNIPPET('network_config')
reboot

rootpw --iscrypted $default_password_crypted
selinux --disabled
skipx
timezone --utc Europe/Paris
install
zerombr
autopart


%pre
$SNIPPET('log_ks_pre')
$kickstart_start
$SNIPPET('pre_install_network_config')
# Enable installation monitoring
$SNIPPET('pre_anamon')

%packages
$SNIPPET('func_install_if_enabled')

%post
$SNIPPET('log_ks_post')
$yum_config_stanza
$SNIPPET('post_install_kernel_options')
$SNIPPET('post_install_network_config')
$SNIPPET('func_register_if_enabled')
$SNIPPET('download_config_files')
$SNIPPET('koan_environment')
$SNIPPET('redhat_register')
$SNIPPET('cobbler_register')
$SNIPPET('post_anamon')
$kickstart_done

:wq
The Cobbler configuration is now finished, we can test the Cobbler service. We create a new VM without boot media, we must activate network/PXE in the boot sequence. Then we start this new VM that will ask an IP address to DHCP server and ask PXE for install support.


COBBLER WEB
We will now install the web interface allows the administration of the cobbler and then configure:
yum install cobbler-web

vi /etc/cobbler/modules.conf

[authentication]
module = authn_configfile

[authorization]
module = authz_allowall

:wq

htdigest -c /etc/cobbler/users.digest "Cobbler"cobbler
# Type cobbler twice when asking password to have login/password: cobbler/cobbler

/etc/init.d/cobblerd restart
/etc/init.d/httpd restart
You can get on the interface with a browser to the URL: http://192.168.0.X/cobbler_web (where X must be replace with your Cobbler IP)
Login: cobbler
Password: cobbler

On this web interface, you can do everything that you can do from command line interface (CLI).


KOAN
Koan is a small program that adds to cobbler and interacts with it for reinstallation and virtualization management. Koan means: kickstart over a network, thus we can use it on a client server to calling cobbler server.
In our case, we will mainly use it for deploying virtual machines. It can be installed via yum, but a more current version is available from the EPEL repo.
Be careful about kernel used in CentOS custom ISO. In my experience, I met problems with some kernels on Xen systems using koan.

To get koan and install it:
cd /downloads
wget http://download.fedora.redhat.com/pub/epel/5/x86_64/koan-2.0.9-1.el5.noarch.rpm
rpm -i koan-2.0.9-1.el5.noarch.rpm
Here is an example of koan command to install CentOS virtual machine on Xen system (kernel 2.6.18-194.el5)
koan --virt --profile=centxenvm-xen-x86_64 --server=192.168.0.98 --virt-type=auto --nogfx --virt-name=centos –virt-path=/home/xen/centos/
I don't specify the method used to deploy a new VM with koan because I had to customize a lot of thing before that the VM works... It's DIY and I'm not quite happy with it in order to describe it !

This howto is ended, I hope that you can apply it in your workspace and that you understand how having a Cobbler server working perfectly. I use a CentOS environment but I think that other distros are supported with this system. Contact me if you need more help with Cobbler.

lundi 18 juillet 2011

phpSysInfo

phpSysInfo is a PHP script that displays information about the host being accessed. It will displays things like :

- Uptime
- CPU
- Memory
- SCSI, IDE, PCI
- Ethernet
- Floppy
- Video Information

website: http://phpsysinfo.sourceforge.net/


vendredi 1 juillet 2011

Innotop: MySQL InnoDB Monitor

Innotop est un clone de top (ou top-like) pour MySQL, orienté innoDB. Cette application vous permet d'effectuer de la surveillance. Par ailleurs, ce projet propose plus de fonctionnalités et de souplesses par rapport aux autres outils de la même famille.

Cette nouvelle version (1.8.0)  apporte de nouvelles fonctionnalités, comme :

  • Une interface en plugins entièrement configurable
  • Une surveillance du nombre de serveurs avec la possibilité de les regrouper
  • Une trousse à outils variés comme mytop
  • De nouvelles fonctionnalités comme l'interrogation de la période
  • Analyse et affiches les informations venant de InnoDB
Lien vers le site Innotop: http://code.google.com/p/innotop/


dimanche 19 juin 2011

htop - an interactive process viewer for Linux

htop est un moniteur système pour Linux, presque le même que top, que nous connaissons tous. Mais qui propose une interface plus conviviale que ce dernier. Il est programmé en C à l’aide de la bibliothèque ‘ncurses’.



Les + de htop (par rapport à top):
+ htop permet de ‘tuer’ un processus directement, sans se soucier de son PID.
+ htop permet, de la même manière, d’augmenter/réduire la priorité d’un processus sans connaître son PID.
+ htop est plus user-friendly, dans ses raccourcis, sa présentation, et le scroll.
+ htop permet un affichage en arborescence, identifiant clairement les processus pères et fils.
+ htop est plus rapide.
+ htop gère la souris.
+ htop intègre une jauge de l’utilisation de la mémoire vive, cpu et swap.


Screenshot:


Le site de htop: http://htop.sourceforge.net/


Un outil indispensable !

WebWait - Benchmark your Website

WebWait est un site permettant de tester votre connexion web ainsi que les performances de votre site. En effet, on va mesurer le temps d'accès au site.


Il suffit d'entrer l'URL du site web et de cliquer sur le bouton "Time it !". On peut préciser le nombre d'appels et l'intervalle entre ces appels pour le bench.
Sur la partie droite, on peut voir un aperçu du site testé.


A la fin du test, le résultat affiché correspond à la moyenne du temps d'accès sur la totalité des appels.
Ceci permettra d'avoir une idée sur les performances du site testé et de constater la fluidité de la navigation coté utilisateur.


Voici le lien vers WebWait: http://webwait.com/


vendredi 10 juin 2011

(R)?ex What do you want to deploy today ?

Let me introduce a new project that will make a lot of noise !


This project is named Rex and it's for lazy sysadmins.
You can manage files, directories, packages, services and many more on all of your servers.


Features are: 
The definition of tasks and profiles is very easy,
You need just a ssh connection, no agent is required,
Run tasks parallel,
Customization is really easy,
Currently supports on Debian, Fedora, CentOS and OpenSuSE


It is based on Perl and you don't need to control perfectly the perl language for playing with Rex.
Current version is 0.6.0


I tried to use fabric (based on python) but it was more difficult to customise tasks and to do what I want.
I find rex and I advise you to use it !


Website is http://rexify.org
You will find all informations there.


IRC on irc.freenode.net channel #rex



jeudi 9 juin 2011

Widget: carte 3D des visiteurs

Voici un widget très fun qui permet de visualiser où sont situés géographiquement les visiteurs de votre site à travers un globe terrestre en 3D.
C'est en plus très simple de mettre ce globe sur son site, il suffit d'aller sur le site http://www.revolvermaps.com/


Vous cliquez ensuite sur "Get Your Globe" et vous allez pouvoir personnaliser les caractéristiques de votre widget en 4 étapes: type de globe, dimensions de celui-ci, couleur des marqueurs et quelques paramètres avancés.
Il ne reste plus qu'à copier le code généré et l'insérer dans la page web de votre site.


Voici un aperçu de ce widget:

vendredi 3 juin 2011

Google version geek

Voici un site qui a déjà fait pas mal de parler de lui mais pour ceux qui ne connaissent pas, ça peut être intéressant. Il s'agit en fait d'une version très geek du moteur de recherche le plus utilisé aujourd'hui.


Un site qui permet de faire des recherches sur Google dans un environnement Shell: 


lundi 7 mars 2011

Customize CentOS



This is the method that I use to customize CentOS x86_64 distro. The objective of this operation is reducing image install used for server. We need those two ISO: CentOS x86_64 netinstall and CentOS x86_64 CD1. CentOS netinstall permits you to install a fresh mini install but there's yet in this way some packages that are useless for using on a server. On netinstall install, where only the vim package is selected , all the following packages are useless for me: wireless-tools xorg-x11-filesystem Deployment_Guide-fr-FR libXcursor rhpl libpng libjpeg libtiff cups-libs bitstream-vera-fonts fontconfig libXft libXext libXcursor libXrandr libXdmcp libXinerama ecryptfs-utils gtk2 libX11 libXi libXrender cairo trousers libXau libXfixes But some others useful packages for server were not installed: slocate autoconf automake rsync make imake m4 man yum-utils elinks telnet First, you have to do this fresh mini netinstall for create a local repository, because customising an ISO, for reducing it, essentially requires the removal of unnecessary packages. So when install is done. We create a mini local repository, with packages installed on this fresh install. Those that will be included into the custom ISO. How to:
# remove useless packages
yum remove wireless-tools xorg-x11-filesystem Deployment_Guide-fr-FR libXcursor rhpl libpng libjpeg libtiff cups-libs bitstream-vera-fonts fontconfig libXft
libXext libXcursor libXrandr libXdmcp libXinerama ecryptfs-utils gtk2 libX11 libXi libXrender cairo trousers libXau libXfixes -y

# install required packages
yum install slocate autoconf automake rsync make imake m4 man yum-utils elinks telnet yum-utils -y

# listing all packages and download them into temp directory
rpm -qa > /home/rpm.log && mkdir /tmp/rpm
for i in $(cat /home/rpm.log) ; do yumdownloader "$i" -y --destdir /tmp/rpm ; done

# To create repo (useless for customize CentOS ISO)
yum install createrepo -y
createrepo /tmp/rpm/
cd /etc/yum.repos.d/

vi local.repo

[localrepo]
name= CentOS Server $releasesever - My Local Repo
baseurl=file:///tmp/rpm/
enabled=1
gpgcheck=0
#gpgkey=file:////etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

:wq
Now packages are downloaded and we can start to build new ISO. To customize CentOS distro, we only need of the first CentOS CD among eight medias CD for normal install. Thus after downloading, mount the iso on this linux server and customise your distro. How to Second part :
# Create directories
mkdir /mnt/cdimages/
mkdir /mnt/cdimages/CentOS-5.5-x86_64
mkdir /mnt/cdimages/CentOS-5.5-x86_64-core

# Mount ISO
mount -t iso9660 -oloop,ro /home/CentOS-5.5-x86_64-bin-1of8.iso /mnt/cdimages/CentOS-5.5-x86_64

# Install needed packages
yum install mkisofs cdrecord

# Rsync ISO content to new CentOS minimal core 
rsync -av --exclude CentOS/ /mnt/cdimages/CentOS-5.5-x86_64/ /mnt/cdimages/CentOS-5.5-x86_64-core/
mkdir /mnt/cdimages/CentOS-5.5-x86_64-core/CentOS
cp -r /tmp/rpm/* /mnt/cdimages/CentOS-5.5-x86_64-core/CentOS/

# Edit comps.xml
cd /mnt/cdimages/CentOS-5.5-x86_64-core
wc -l repodata/comps.xml # 11418 repodata/comps.xml
grep "xml:lang" repodata/comps.xml | wc -l # 8811
grep -v "xml:lang" repodata/comps.xml > /tmp/no-lang.comps.xml
wc -l /tmp/no-lang.comps.xml # 2607 /tmp/no-lang.comps.xml
rm repodata/* -f
cp /tmp/no-lang.comps.xml repodata/comps.xml

# Test RPM dependencies
cd /mnt/cdimages/CentOS-5.5-x86_64-core/CentOS
mkdir /tmp/testrpmdb
rpm --initdb --dbpath /tmp/testrpmdb/
rpm --test --dbpath /tmp/testrpmdb/ -ivh *.rpm

echo $? # 0
du -sh /tmp/testrpmdb/ # 644K /tmp/testrpmdb/
rm -rf /tmp/testrpmdb

# Create repo
yum install createrepo -y
cd /mnt/cdimages/CentOS-5.5-x86_64-core
cat .discinfo
export discinfo=$(head -1 /mnt/cdimages/CentOS-5.5-x86_64-core/.discinfo)
echo $discinfo
createrepo -u "media://$discinfo" -g repodata/comps.xml /mnt/cdimages/CentOS-5.5-x86_64-core

# Create bootable ISO
export isofilename=/mnt/cdimages/CentOS-5.5-x86_64-core.iso

mkisofs -r -R -J -T -v \
-no-emul-boot -boot-load-size 4 -boot-info-table \
-V "CentOS-5u5-core 64 bit" -p "CentOS_Custom" -A "CentOS-5u5-core 64 bit-2011/01/01" \
-b isolinux/isolinux.bin -c isolinux/boot.cat -x "lost+found" \
-o $isofilename /mnt/cdimages/CentOS-5.5-x86_64-core 
# 213685 extents written (417 MB)

# Implant MD5SUM in the ISO file
yum install mkcdrec
updatedb
locate implantisomd5
/var/opt/mkcdrec/bin/implantisomd5 $isofilename

## Use New CentOS ISO: /mnt/cdimages/CentOS-5.5-x86_64-core.iso
You can now try to install new virtual machine with this new ISO media boot to check that everything works perfectly.