Upgrading Your Server from Red Hat Linux 9.0 to CentOS 3.x August 6, 2006: Initial Release. August 27, 2006: New version of CentOS 3.x (3.8). Fixed minor typos. April 22, 2007: Updated with new cPanel approved version of Perl 5.8.8. Preamble: Red Hat Linux 9.0 is an old operating system and is no longer actively maintained by Red Hat. Thankfully, others have stepped up to the plate to help keep RH9 secure by updating the legacy packages for that OS whenever new security issues are found. If you have a server with RH9 on it, there's nothing wrong with it. However, like Windows 98 there will come a day when the OS will fade into history. Newer operating systems can provide additional security, stability and features. Red Hat now has two main operating systems, Fedora Core (FC) and Red Hat Enterprise Linux (RHEL). Fedora Core is free and designed for desktop OS systems. It comes with a very short actively supported life span (6 months) and is used by Red Hat to test new features for its bread and butter: Red Hat Enterprise Linux. RHEL is all about stability and a very long (5 year) active support time. RHEL is designed for use by businesses on their mission-critical servers. RHEL is NOT free and can cost businesses quite a lot, especially if they want direct regular support from Red Hat. It should be immediately obvious which sort of OS is best for dedicated servers. Words like "testing" and "short life span" don't exactly make FC the ideal choice. RHEL is the clear winner for servers (we're only comparing Red Hat Linux products, not other OSes). There is a free alternative to Red Hat Enterprise Linux called CentOS. CentOS is the name for an open source OS that takes the copyrighted and non-open source code out of RHEL and replaces it with 100% byte compatible open source code. In truth, most of what they strip out are Red Hat logos and other corporate identity items. It is not maintained by Red Hat, nor does any one company provide support. However, you can learn more about CentOS here: http://centos.org/ In case you are wondering, all of Hostdime's newest servers use CentOS by default. It is rock-solid and ideal for busy server environments. This tutorial is going to cover upgrading a RH9 server to CentOS 3.x. You may notice that CentOS 4.x is the latest version of that operating system, and yet this tutorial covers installing CentOS 3.x. The reasons for this are fairly simple. RH9 and CentOS 3.x share a lot of the same basic architecture and you can upgrade RH9 to CentOS 3.x with nothing more than some time, an Internet connection and root access to the server via SSH. CentOS 4.x uses the newer 2.6.x kernel branch and so you CANNOT upgrade to CentOS 4.x from 3.x or RH9 in the same way described here. If you want to go from RH9 to CentOS 4.x, contact the helpdesk for assistance. It CANNOT be done remotely without some special hardware. So what will you gain from moving from RH9 to CentOS 3? 1) CentOS 3 matches RHEL's own actively supported lifespan of 5 year, so it is still being directly supported. 2) The updated OS components will help increase security 3) Better stability and CentOS can take better advantage of multiple core, multiple CPU servers Things to consider before taking the plunge: 1) This process takes time and will require at least one, but possibly two or more server restarts. This means at least some guaranteed downtime for your customers. 2) If something goes wrong the helpdesk may have to step in an assist as you won't have physical access to the hardware if it won't reboot. 3) The process requires very careful attention to detail! Don't rush or you will skip a step and crash your server. No joke! 4) If your server is not currently running well, is overloaded or you are having problems of ANY kind with your server, DO NOT attempt the upgrade! Wait until load is low and things are running well before upgrading. 5) While this process typically will not result in any lost data, you should be prepared for that. Back up ALL critical data before beginning. 6) This process can take a long time. Up to 4 hours in some cases (though on faster hardware with few issues you may be done in less than an hour). 7) There is no easy way to go back to RH9 if something goes wrong unless you have a complete backup of all data on your server and have it all restored. 8) You need to use root SSH for almost all of this process. There is no substitute. If you feel uncomfortable with the root shell, DO NOT bother with the upgrade. 9) There is no way for me to anticipate every problem you might have, so you have to be able to roll with the punches. Hopefully there won't be any serious problems not covered here. 10) You can't follow these directions on a VPS server, only a dedicated server. 11) Hostdime is NOT responsible for issues you experience with your server if you attempt this. We'll help if we can, but we make no promises that you won't loose data! If your server explodes in a ball of flame or steals your boy/girlfriend after the upgrade (because CentOS is so sexy), don't blame us! If you can't live with the risk, stay with RH9, you'll be OK. Tools you will need before you begin: 1) A printout of this tutorial to refer to. 2) Root access to your server via SSH using whatever tools you prefer. 3) Lots of time and patience. Part 1: Make sure you are using the latest Red Hat 9 kernel and latest cPanel After backing up all of your data and picking a time when you have plenty of uninterupted time to work (you can't stop this process once it is underway), we need to first prepare by making sure the server is running the latest real Red Hat provided 2.4.x kernel. The kernel is the heart of the operating system and handles a lot of low-level operations. As of this writing, the latest RH9 kernel is 2.4.20-31.9 #1. What you need to understand is that a kernel can be compiled with lots of different settings and features enabled or disabled. If you don't use a custom kernel, the typical Red Hat and CentOS kernel types are: kernel (the standard, single non-hyperthreaded processor kernel. If you have problems with any other types, try this one.) kernel-smp (SMP stands for Symetric MultiProcessing. This means the kernel is designed to take advantage of more than one phsyical CPU or a single CPU with multiple cores or hyperthreading) kernel-devel (This is a developmental kernel and may have special settings or test code enabled. Not a good idea for any server) kernel-hugemem (This kernel is designed for use on servers with massive amounts of memory (16 GB or more). Unless you've had us build such a monster for you, you don't need this.) In addition there are kernel builds for Intel and AMD processors, 386, 586, 686 architectures, 64-bit processors, etc.). In any case, your server is probably running either the standard kernel or if it has a hyperthreaded CPU or multiple CPUs, then it is running the SMP version. So how do we know what sort of kernel your server is running? Take the time now to log in two times to your server as root via SSH. Why two times? The second time provides some insurance if something goes wrong. Once in, type: uname -a You'll probably see a line something like this: Linux servername 2.4.20-31.9 #1 Tue Apr 13 18:04:23 EDT 2004 i686 i686 i386 GNU/Linux What this tells you is that you are running Linux, on whatever your server name is and the kernel is non-SMP version 2.4.20-31.9 #1, compiled on Tuesday, April 13, at 6:04 PM Eastern Daylight Savings time (GMT-6), 2004. The architecture is Intel 386/686 Linux. The server this example was taken from was running a Pentium 4 2.6 Ghz single CPU with the latest RH9 kernel at the time. If you have more than one CPU, you should see the letters SMP somewhere after the kernel version. You can also see this and other information in WHM under Server Information. We also need to make sure that RH9 isn't missing any crtical components. We can do this by running up2date, the Red Hat Network automated update process (similar to Windows Update or Mac OS X Software Upate). If you've never use RH up2date before, the process is simple, but you might have to sign up for a free account first. Let's configure up2date properly first. Type: up2date --config This will cause a long series of options to be displayed on your screen. Look for "pkgSkipList" in the display. It sometimes appears in different locations. It might be #8, #20 or #40. Whatever it is, note the number next to it. Type that number in now and press Enter/Return. You will see something like this: Attribute: pkgSkipList Comment: A list of package names, optionally including wildcards, to skip Current value: perl;php*;mysql;httpd;apache;mod_*;exim;courier;squirrelmail; < return for default, C to clear list, items are ';' separated > New Value: No matter what you see liste in Current value, you should type exactly the following into the New Value section: perl;php*;mysql;httpd;apache;mod_*;exim;courier;squirrelmail; When done, press Enter/Return to save the changes. Next look for "removeSkipList" in the list of options and type in the number next to that item. You should see: Attribute: removeSkipList Comment: A list of package names, optionally including wildcards that up2date will not remove Current value: kernel*; < return for default, C to clear list, items are ';' separated > New Value: Make sure the current value is kernel*; If it isn't' type that into New Value and press Enter/Return. If it is, Just press Enter/Return to accept the Current value. When done, press Enter/Return to save all changes and exit config. At this point you might see a warning like this: No valid key selected. Run rpm --import /usr/share/rhn/RPM-GPG-KEY (the exact wording may be different.) If you see this, type the following: rpm --import /usr/share/rhn/RPM-GPG-KEY You should see absolutely NOTHING happen after you execute the command. This is OK. Now we are ready to update the OS or sign up for a new account. Either way, type: up2date -u If the screen clears and turns blue, your server isn't registered with the RHN and we need a free account. Use TAB and the arrow keys to move around the screen. When prompted create a username and password and enter a valid e-mail address for the account. You DO NOT have to enter any physical address information if you do not want to. Accept all other defaults. Keep selecting NEXT. Your profile will be sent to RH and you will receive an e-mail to whatever e-mail address you gave telling you two things: 1) RH9 is no longer supported (end of life). 2) Please confirm your e-mail address. Do so. After the profile is sent to RH, your server will automatically connect to the RHN and update any old or missing files (including your kernel). If you see that the kernel is being updated, we will need to configure your server to use this new kernel by default. If the updates do NOT include any kernel files, then you can skip this part. One thing before we do that, however. If you know you have multiple processors or multiple cores in your CPU and your server isn't using the SMP version of the kernel and it hasn't just downloaded kernel-smp you will want to do so. Normally you should NOT need to do this, but in some cases the OS doesn't realize that you are using more than one CPU. In this case after the update finishes type: up2date kernel-smp and let it be installed if it isn't already. Now on to choosing the correct kernel for bootup. There are two different kinds of bootloaders on most RH servers. Grub and lilo. Grub is used by default, so we'll just configure that. Rather than use the command "grub" which can take 5 minutes or more to find all the various hardware settings before you can configure it, we will go an edit the grub configuration file directly: pico -w /etc/grub.conf (or your server may use /boot/grub/grub.conf or /boot/grub.conf or perhaps some other location, if so, be sure to edit the correct file or all of them if you are unsure) You'll see something like this: # grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/hda5 # initrd /initrd-version.img #boot=/dev/hda default=0 timeout=10 splashimage=(hd0,0)/grub/splash.xpm.gz title Red Hat Linux (2.4.20-31.9) root (hd0,0) kernel /vmlinuz-2.4.20-31.9 ro root=/dev/hda5 initrd /initrd-2.4.20-31.9.img title Red Hat Linux (2.4.25-ow1) root (hd0,0) kernel /vmlinuz-2.4.25-ow1 ro root=/dev/hda5 timeout=10 splashimage=(hd0,0)/grub/splash.xpm.gz title Red Hat Linux (2.4.20-31.9) root (hd0,0) kernel /vmlinuz-2.4.20-31.9 ro root=/dev/hda5 initrd /initrd-2.4.20-31.9.img title Red Hat Linux (2.4.25-ow1) root (hd0,0) kernel /vmlinuz-2.4.25-ow1 ro root=/dev/hda5 title Red Hat Linux (2.4.20-8) root (hd0,0) kernel /vmlinuz-2.4.20-8 ro root=LABEL=/ hdb=ide-scsi initrd /initrd-2.4.20-8.img You may have more or less in your grub.conf file. What you want to look for is the latest RH kernel version listed in the "title" sections. In the above example, it is the first set of entries: splashimage=(hd0,0)/grub/splash.xpm.gz title Red Hat Linux (2.4.20-31.9) root (hd0,0) kernel /vmlinuz-2.4.20-31.9 ro root=/dev/hda5 initrd /initrd-2.4.20-31.9.img Keep in mind that if you are using multiple CPUs or hyperthreading or dual cores, you shoul look for a similar line that mentions SMP. That will be the one you want to use. To set this kernel as the default, we need to set the "default=#" (where # is replaced with some number) to whatever title line is the correct and latest kernel. The numbering starts from 0 (zero) for the first title line, 1 for the second group, 2 for the third and so on. In the example, there are three different kernels on this machine. Two old ones and the latest one. Hopefully the very latest kernel will be the first group in the list (as it is above). If so, the line "default=#" should be set to 0 (zero). Once you've set the correct default number, save changes to the file. Now you will have to restart the machine to load the new RH kernel (if you aren't already running it). If you are already running the latest RH kernel, you can SKIP THE REBOOT STEP! To reboot your server you have several choices. If you prefer to do it in WHM, make sure you select the Graceful Reboot option. If you do it in SSH, you can do it one of a few ways. The two I prefer are: reboot or shutdown -r # (where # is any number) The shutdown command with the -r switch reboots the server in the number of minutes you specify after the -r. DO NOT forget the -r switch or your server will turn itself off rather than reboot. The shutdown command is nice because it announces (using wall) to everyone logged into your server via SSH that the server will shut down in # minutes, and count down as the reboot gets closer. The reboot command is simple and does the trick, but without the countdown. Log out if you have the time. How long the reboot takes depends on your server hardware and whether FSCK (the Linux equivalent of disk check) is being run or not. If FSCK is run, the process can take 10-45 minutes in some cases, and during this time your server will be offline. Most of the time your server won't need to run extended FSCK tests and the server should be back online within 5 minutes or so. If it doesn't come back online, contact the helpdesk for assistance. Once your server is back online and running the latest RH kernel, we should update cPanel so that it is running the latest version (this will also make sure critical files aren't missing for cPanel operation): /scripts/upcp After that, there is one more thing you should run: rpm -vv --rebuilddb This will rebuild the RPM database that up2date (and later yum) use. This process shouldn't take too long (perhaps 1 minute), but if your server is overloaded it could take longer. When it is finished, if you've gotten no other errors, we can begin the actual installation process. Part 2: Install Yum and CentOS First we need to install YUM which is the automatic update mechanism used by CentOS (instead of up2date, which, while it isn't used will remain on the server for compatibility). Yum requires one package which isn't installed by default on RH9, so we will install it first. Type: up2date install libxml2-python This will install the libxml2-python package if it isn't already installed. If you get any errors like: error: db4 error(-30989) from dbcursor->c_get: DB_PAGE_NOTFOUND: Requested page not found You can ignore them. However, if libxml2-python cannot be installed, you should attempt to fix whatever issue you have before continuing. Once that is done we can install yum: cd wget http://linux.duke.edu/projects/yum/download/2.0/yum-2.0.8-1.noarch.rpm rpm -ivh yum-2.0.8-1.noarch.rpm This should complete without any issues. Now we have to configure the yum.conf file so that we can get files from the CentOS repository. pico /etc/yum.conf You will find several lines already in this file. Delete them (CTRL-K in pico and nano will delete the line under the cursor). Paste in the following to the now empty yum.conf file: [main] cachedir=/var/cache/yum exclude=mod_ssl* httpd* perl mysql* php* spamassassin* squirrelmail courier* apache* exim* mod_* debuglevel=2 logfile=/var/log/yum.log pkgpolicy=newest distroverpkg=redhat-release installonlypkgs=kernel* tolerant=1 exactarch=1 [base] name=CentOS-$releasever - Base #baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/ baseurl=http://mirror.centos.org/centos/3.8/os/$basearch/ gpgcheck=1 #released updates [update] name=CentOS-$releasever - Updates #baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/ baseurl=http://mirror.centos.org/centos/3.8/updates/$basearch/ gpgcheck=1 #packages used/produced in the build but not released [addons] name=CentOS-$releasever - Addons #baseurl=http://mirror.centos.org/centos/$releasever/addons/$basearch/ baseurl=http://mirror.centos.org/centos/3.8/addons/$basearch/ gpgcheck=1 #additional packages that may be useful [extras] name=CentOS-$releasever - Extras #baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/ baseurl=http://mirror.centos.org/centos/3.8/extras/$basearch/ gpgcheck=1 #additional packages that extend functionality of existing packages [centosplus] name=CentOS-$releasever - Plus #baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/ baseurl=http://mirror.centos.org/centos/3.8/centosplus/$basearch/ gpgcheck=1 #packages in testing #[testing] #name=CentOS-$releasever - Testing #baseurl=http://mirror.centos.org/centos/$releasever/testing/$basearch/ gpgcheck=1 Save changes when done. Before we can start the update, we need to import the CentOS 3.x GPG key. Run: rpm --import http://mirror.centos.org/centos/3/os/i386/RPM-GPG-KEY-CentOS-3 You won't get any output from this command if all goes well. Now we start the CentOS installation process right over RH9 on our live server. Make sure you've got another SSH connection open to your server and root WHM loaded in your web browser in case something goes wrong. Or you can use screen if you are familiar with it and it is installed. Start the upgrade by typing: yum upgrade Yum does most of the work from here. It downloads RPM package data for CentOS, sets up local repositories, looks at the existing system and tries to find things that need to be updated to bring the OS up to the latest version of CentOS 3.x. Note as of this writing CentOS 3.8 is the latest version of CentOS 3.x. You will see on-screen that it looks like yum is fetching CentOS 9. It isn't. That 9 is coming from RH9 which is currently installed. The yum config file is set to fetch CentOS 3.8, so that is what it will actually get. Don't go anywhere. The initial process of yum looking for files to update takes a few minutes, but you will have to answer a question before the actual install begins. Eventually, yum will list all the packages that need updating and ask you: Is this ok [y/N]: Type "y" when you are ready to begin the actual upgrade. From this point on you must be SURE your server does not get rebooted or the process get interrupted or you could corrupt the OS. It will begin to download files to your server. When that is done, the files will be installed and configured and then cleaned up. After the download of the packages, the exact number of steps that yum has to execute before it is finished will be displayed as each step is done. This number varies, but expect it to be somewhere around 500. The exact number depends on how your server is configured and what packages are installed. This process can take an hour or more, though on fast servers it could be finished in 30 minutes. Be patient and keep an eye on things. Very few things should go wrong and most "warnings" aren't anything at all to worry about. Things like: warning: /etc/issue saved as /etc/issue.rpmsave warning: /etc/issue.net saved as /etc/issue.net.rpmsave Just are there to let you know that the file was already configured, so the file from the new package (that isn't configured) was saved with a different name in that location so if you want you can later compare the two files for changes. Generally, just ignore them. The only serious problems are if yum tells you it cannot proceed because it is missing critical files (which you will have to install before continuing the upgrade process) or if you get disconnected from your server for any reason. If you do get disconnected, don't immediately panic. The upgrade process may still be active on your server. If so, just fall back on your other SSH login and run ps aux or top to see if "yum" is still active. If so, let it go, it is still working. Watch the yum process to see if it continues, if not try running yum upgrade again. If all of your SSH connections are disconnected, go into WHM and reboot OpenSSH. This will probably bring SSH back online so you can log in again. If everything goes bad and nothing is working, contact the helpdesk for assistance. Your data backups will come in handy. If you've gotten yum to complete the upgrade successfully, and if you reload WHM, you should see "CentOS 3.7" instead of "Red Hat 9" in the upper-right corner. We're not done yet, though. Part 3: Post install configuration, server reboot and final configuration It wouldn't be a bad idea to run yum upgrade again one more time after it finishes, just to be sure that it really did upgrade everything. Unless there were some problems, yum shouldn't find anything else to upgrade. There are a few other things to check before we restart the server for the last time. First, we need to see if the CentOS kernel was installed properly. To do that, you can run: rpm -qa|grep kernel This should turn up a list of all of a few items. You want to look for something like this (though on your server it may be slightly different): kernel-2.4.21-47.EL Now we need to be absolutely certain the new CentOS 2.4.x kernel will load by default on reboot, so we go back to /etc/grub.conf: pico -w /etc/grub.conf Look for something like this (with SMP if your server needs it): splashimage=(hd0,0)/grub/splash.xpm.gz title CentOS (2.4.21-47.EL) root (hd0,0) kernel /vmlinuz-2.4.21-47.EL ro root=/dev/hda5 initrd /initrd-2.4.21-47.EL.img Just as before, look for the title line that mentions CentOS and choose the correct kernel for your server (look for SMP if you have more than one CPU, for example). Normally, the correct CentOS kernel will be the top most group. Just make sure the "default=#" is set to the correct group (remember zero is the first kernel in the list, 1 is the second, and so on). If you make changes, save the changes and exit. Now you can restart your server (via WHM, graceful reboot, or via SSH reboot or shutdown -r #. The server will go offline and load up with the new CentOS 3.x kernel. The downtime depends on your server, but it probably won't be more than 10 minutes. If it doesn't come back online, contact the helpdesk for assistance. Once it is back online, we should make a few changes to ensure everything will work well. First we need to reconfigure yum.conf to dynamically look for updates (and not always look at the 3.7 branch): pico -w /etc/yum.conf To make sure you make the proper edits, let's just remove the old yum.conf options and replace them with the new ones. Use CTRL-K or whatever other method you want to clear out the file (you could also rename it and create a new yum.conf file if you prefer). Paste in the following lines: [main] cachedir=/var/cache/yum exclude=mod_ssl* httpd* perl mysql* php* spamassassin* squirrelmail courier* apache* exim* mod_* debuglevel=2 logfile=/var/log/yum.log pkgpolicy=newest distroverpkg=redhat-release installonlypkgs=kernel* tolerant=1 exactarch=1 [base] name=CentOS-$releasever - Base baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/ #baseurl=http://mirror.centos.org/centos/3.8/os/$basearch/ gpgcheck=1 #released updates [update] name=CentOS-$releasever - Updates baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/ #baseurl=http://mirror.centos.org/centos/3.8/updates/$basearch/ gpgcheck=1 #packages used/produced in the build but not released [addons] name=CentOS-$releasever - Addons baseurl=http://mirror.centos.org/centos/$releasever/addons/$basearch/ #baseurl=http://mirror.centos.org/centos/3.8/addons/$basearch/ gpgcheck=1 #additional packages that may be useful [extras] name=CentOS-$releasever - Extras baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/ #baseurl=http://mirror.centos.org/centos/3.8/extras/$basearch/ gpgcheck=1 #additional packages that extend functionality of existing packages [centosplus] name=CentOS-$releasever - Plus baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/ #baseurl=http://mirror.centos.org/centos/3.8/centosplus/$basearch/ gpgcheck=1 #packages in testing #[testing] #name=CentOS-$releasever - Testing #baseurl=http://mirror.centos.org/centos/$releasever/testing/$basearch/ gpgcheck=1 Save changes and exit pico/nano. Note that what we've really done is comment out the old baseurl line and uncomment the other line. This will make sure CentOS always keeps itself properly updated, even if CentOS 3.9 is released. Now run yum upgrade one last time to be sure there aren't any newer updates. Then run: yum clean to get rid of some old files. There are two other things I strongly recommend you do before you call it a night. First, let's make sure you're using the latest cPanel approved version of Perl. Type: perl -v If you see any perl version other than 5.8.8, then we need to install cPanel's approved perl version. cd wget http://layer1.cpanel.net/perl588installer.tar.gz tar xfz perl588installer.tar.gz cd perl587installer ./install This will download and install Perl. It may take 10-20 minutes. When done, remove the perl installer files: cd rm -Rf perl588* When it is done, update cPanel to make sure it has all of the correct install versions of the files it needs. Do this even if you didn't need to install Perl! /scripts/upcp --force Once done, you've successfully made the switch. Congratulations! Enjoy your new OS! Note that you will eventually get an e-mail warning you that your server will be removed from the Red Hat Network since it hasn't connected to update RH9 in a long time. This isn't a problem and won't affect CentOS at all.