I'm about to upgrade one of our HDR pairs from a relatively old version of 11.50 to the most recent drop to resolve a minor bug. Since an upgrade is something you may be interested in doing some day, I thought it would be a good idea to document my steps here.
Normally an upgrade from xC3 to xC8 would be trivial and low risk since we're only upgrading the minor versions, but with the addition of Compression in xC4 any upgrade from xC3 or lower must be treated as a major upgrade which includes a reinitialization of HDR.
I want to minimize downtime by doing as much of the work as I can outside of the maintenance window and I want to minimize my risk by testing the following migration and reversion steps on my test systems to ensure there are no gotchas and choreograph my upgrade in a way that always gives me a quick and safe way to abort mission and get production up and running if something goes wrong during the upgrade.
Lucky for you and me, the bulk of the upgrade work can be accomplished during normal business hours with no impact or downtime. Not only does this minimize the amount of late night/early morning maintenance window work you will have to do, it will minimize the mistakes that can happen when you are doing work during sleepy time.
A lot of my upgrade steps are based on INFORMIXDIR being a link to an actual directory, you did remember to do that didn't you? If not, you should invest the time and energy required to make INFORMIXDIR a link, you will be glad you did.
Get yourself some binaries
There are two types Informix tar balls, there is the bundle which includes everything (engine, CSDK, iConnect, etc.) which is typically downloaded from Passport Advantage or the Innovator-C download page and there are Fix Packs which are typically downloaded from Fix Central and only include the engine.
The bundle is installed via ids_install and the engine is installed via installserver if that helps.
If upgrading with a Fix Pack, you will need the bundle used for your current installation. For this upgrade I am using a Fix Pack for 11.50.FC8W2 and have the 11.50.FC3 bundle from my initial installation.
Install the new engine - Perform the following steps on the HDR Primary and HDR Secondary
Let us skip some of the boring stuff and say that I have downloaded my Fix Pack, located my initial bundle and unpacked each tar-ball into their own temporary folder.
I like to install Informix into a version specific directory, for example /opt/informix-ids.11.50.FC8W2, and use the soft link /opt/informix as my INFORMIXDIR. This allows me to pre install everything and then simply shutdown the engine, redirect the link and restart the engine when it is time to upgrade.
I will first install the FC3 bundle into the 11.50.FC8W2 directory by temporarily setting INFORMIXDIR to /opt/informix-ids.11.50.FC8W2 and running ids_install as root.
root:/tmp/ids-11.50.FC3> mkdir /opt/informix-ids.11.50.FC8W2 root:/tmp/ids-11.50.FC3> chown informix:informix /opt/informix-ids.11.50.FC8W2 root:/tmp/ids-11.50.FC3> export INFORMIXDIR=/opt/informix-ids.11.50.FC8W2 root:/tmp/ids-11.50.FC3> ./ids_installAfter this is complete I will install the FC8W2 Fix Pack on top of the FC3 bundle as root.
root:/tmp/ids-11.50.FC8W2> export INFORMIXDIR=/opt/informix-ids.11.50.FC8W2 root:/tmp/ids-11.50.FC8W2> ./installserver
Copy configuration files from current install directory to new install directory - HDR Primary and Secondary
We will need to copy some files from the current INFORMIXDIR to our soon to be new INFORMIXDIR. This is what I need to copy, your needs may vary slightly.
# definitely will need the ONCONFIG file cp /opt/informix/etc/$ONCONFIG /opt/informix-ids.11.50.FC8W2/etc/ # and we'll need this file cp /opt/informix/etc/oncfg_* /opt/informix-ids.11.50.FC8W2/etc/ # I setup my HDR trusts in the Informix hosts.equiv file cp /opt/informix/etc/hosts.equiv /opt/informix-ids.11.50.FC8W2/etc # if you are using a storage manager for backups, don't forget this guy cp /opt/informix/etc/sm_versions /opt/informix-ids.11.50.FC8W2/etc # if your sqlhosts file lives somewhere in INFORMIXDIR, don't forget to copy it cp /opt/informix/etc/sqlhosts /opt/informix-ids.11.50.FC8W2/etc # and finally the alarmprogram if you have made any changes cp /opt/informix/etc/alarmprogram.sh /opt/informix-ids.11.50.FC8W2/etcJust one more thing to do to make sure we are protected
Take a Level 0 backup, it is better to have it and not need it than to need it and not have it.
We are now ready to upgrade
We will break HDR by converting the Primary and Secondary nodes to standard engines and we will shut down the Secondary node and not touch it until we are 100% satisfied the upgrade was successful. This gives us a quick way to back out of our upgrade attempt if something goes wrong. If there is a problem we can restart the Secondary (which is now a writable standard engine) and direct the clients to this node while we figure out what went wrong.
Convert the Primary to a standard engine and shutdown
informix@primary> onmode -d standard informix@primary> onmode -kyConvert the Secondary to a standard engine and shutdown
informix@secondary> onmode -d standard informix@secondary> onmode -kyRelink /opt/informix on the Primary
informix@primary> rm /opt/informix informix@primary> ln -s /opt/informix-ids.11.50.FC8W2 /opt/informixStart the upgraded engine on the Primary
informix@primary> oninit -vIf everything goes as planned the upgrade will complete successfully and your primary engine will only be down for a few minutes and your downtime will be minimized.
You can use this time to verify that everything with the upgrade has gone as planned. Check the online.logs for any strange errors and watch your applications for any unexpected behaviour. When you are confident that the upgrade was successful you should take another Level 0 backup, upgrade the Secondary and rebuild HDR.
Relink /opt/informix on the Secondary and reinitialize HDR
informix@secondary> rm /opt/informix informix@secondary> ln -s /opt/informix-ids.11.50.FC8W2 /opt/informix
Follow the instructions for initializing HDR from a previous post.
Oh Noes! Something went wrong. REVERT!
So you thought everything was hunky dory with the upgrade but after reinitializing HDR you realize you need to revert back to your previous version. Gross.
Thankfully Informix has a utility to help with this, the -b switch of the onmode utility.
> onmode -b -- You may only revert IBM Informix Dynamic Server to IBM Informix-OnLine Version 7.31.UC1 or later: Use 'onmode -b 7.3' to revert to 7.3x Use 'onmode -b 9.2' to revert to 9.2x Use 'onmode -b 9.3' to revert to 9.3x Use 'onmode -b 9.4' to revert to 9.4x Use 'onmode -b 10.00' to revert to 10.00x Use 'onmode -b 11.10' to revert to 11.10x Use 'onmode -b 11.50.xC1' to revert to 11.50.xC1 Use 'onmode -b 11.50.xC2' to revert to 11.50.xC2 Use 'onmode -b 11.50.xC3' to revert to 11.50.xC3 Use 'onmode -b 11.50.xC4' to revert to 11.50.xC4 Use 'onmode -b 11.50.xC5' to revert to 11.50.xC5 For more information refer to Section II of IBM Informix Migration guideConvert the Primary to a standard engine
informix@primary> onmode -d standardConvert the Secondary to a standard engine and shutdown
informix@secondary> onmode -d standard informix@secondary> onmode -kyExecute the onmode -b command and revert disk structures back to 11.50.xC3
Informix will be offline after a successful onmode -b
informix@primary> onmode -b 11.50.xC3Relink /opt/informix to the old INFORMIXDIR
informix@primary> rm /opt/informix informix@primary> ln -s /opt/informix-ids-11.50.FC3 /opt/informixStart the engine in Quiescent mode and verify everything is Ok
informix@primary> oninit -sBring the engine into online mode
informix@primary> onmode -mFinally you will want to complete a Level 0 backup and reinitialize HDR.