Stout Upgrade Details 2010This is a featured page

This year's upgrade process will be in 6 parts.

1. Setup the read-only mode copy of Stout's Voyager (outage early in the morning) (outage done at 4:45am)
2. Patch Oracle to 10.0.2.4 and apply CPU (5:28am files copied - picking up here after a short 1-2 hour break)
3. Upgrade Voyager to 7.2.0
4. Apply Voyager Service Pack to get to 7.2.1
5. Do all split server configuration work
6. Stout library tests and customizes as needed, then switch back production (short outage).

Read Only Mode (starting at 2am - up at 4:45am)
- Take down production (3:30am - completed)
- Comment out cron jobs (completed)
- Configure UPG zones on widener (3:36am - completed)
- Copy zones from production to UPG zones on bodleian (completed)
- Copy backups to UPG zones (completed)
- Bring up UPG zones as production (in progress - 4:31am)
- Verify all is well (completed)
- Set Voyager read-only mode (completed)
- set Oracle tablespace to read-only (completed)
- Modify production zones to use UPG ip addresses and check firewall rules (completed 4:54am)
- Oracle listener configuration (completed 4:54am)
-- /oracle/app/oracle/product/10.2.0/db_1/network/admin/listener.ora backup and change
- /etc/inet/hosts file (completed 4:54am)
- Make a backup copy of production web zone /m1/voyager/xxxdb to be referred to for customizations later (completed 5:00am)

Patch Oracle to 10.0.2.4 and apply CPU (completed - 7:30am to 11:45am)
bw - taking a short hour or two break here - 5:30am
- Copy oracle patch files to /m1/incoming/oracle (completed 7:45am)
- Uncompress (completed 7:55am)
- Pre-installation steps (in progress - 8:14am - step 2.1.4 - need to retune Oracle - completed 10:15am)
- Install 10.2.0.4 update (completed 10:41am)
- Run upgrade script and recompile script (inprogress - 10:45am - completed at 11:45am)
- Install Oracle CPU (not doing at this time)

Install Voyager 7.2.0 upgrade (in progress - 12:10pm - completed 2:46pm)
- Copy the voyager upgrade files to /m1/incoming/v720 (completed)
- Run 7.2.0 upgrade
- Prep -
- install new jdk from SHARED tarball (completed)
- verify /m1/voyager/xxxdb/ini/voyager.env has JAVA_HOME set correctly (completed - need to reverify after upgrade.pl)
- move /m1/shared/apache2 to /m1/shared/httpd/2.2.6 and make a symlink from /m1/shared/httpd/2.2.6 to /m1/shared/apache2 (completed on db and web)
- change /m1/voyager/xxxdb mountpoint so it's not a mountpoint for the upgrade.pl run (completed 5:19am)
- Files upgrade.pl (completed 1:02pm)
- Verify /m1/voyager/xxxdb/tomcat/vxws/conf/server.xml entries - (completed)
-- <Valve className="org.apache.catalina.valves.RemoteAddrValve" allow="128.104.22."/> --> (completed)
-- url="jdbc:oracle:thin:@HOSTNAME:PORT:VGER" (completed)
- RE-verify /m1/voyager/xxxdb/ini/voyager.env has JAVA_HOME set correctly (gone - fixed - completed)
- Oracle schema update (completed 1:11pm)
- (Run regen) - not this year
- Post setup steps (in progress)
-- UB (completed)
-- Media scheduler (completed)
-- Authorities Keyword stuff (completed)
-- Run through full Authority Headings piece, including ctxsys recreation and running ./create_single_index.ksh $USERPASS (completed)
-- recreate readonly user (completed)
- Service re-setup - make sure the /m1/voyager/xxxdb directories are only the valid ones or the voyager service setup will do funny things with the extras (completed)

Install Voyager 7.2.1 service pack (in progress 2:50pm - completed 3:55pm)
- Install service pack on DB server (completed)
- shut down zones and change /m1/voyager/xxxdb mountpoint so it is a mountpoint again and rsync the new xxxdb directory over to it (completed)

(bw - going to break here for some real food, will get going with the server splits after dinner tonight -~4pm)
Post Service pack server splits (completed 10:05pm)
- Split web server - don't use maketomcatwebserver.ksh method (completed 9:17pm)
-- verify upgraded java jdk to 1.6 version
-- cleaned up older bin/lib/utility/sbin directories
-- built tarball with -
--- sudo tar cvf webserver.tar stoutdb/etc stoutdb/webvoyage stoutdb/ini stoutdb/tomcat/vwebv lib/2007.2.1 bin/2007.2.1/*exe bin/2007.2.1/*cgi
-- sudo vi *db/etc/webvoyage/voyager.ini
-- under xxxdb/tomcat/vwebv find the WEB-INF/web.xml and verify XServiceHost is pointing to the right place
-- start things up
-- went slower than it needed to because I thought it was going to be harder

- Split z3950 server (completed, other than switch over change)
-- with prod switch over, will need to modify for prod IPs and stop/start voyager on z3950
--- /m1/voyager/stoutdb/etc/ascopac/opac.ini
--- /m1/voyager/stoutdb/ini/z3950svr.ini

- Split staff client server (not completed - going to leave it without for now - split out the group instances and Milwaukee)
-- build zone/create users/groups/home directories, etc.

Customize upgraded Webvoyage and test staff clients (completed)
- Use the stolibupg.wisconsin.edu address for customizations
- Copy of pre-upgrade stoutdb directory should be /m1/voyager/v710-stout
- Point upgraded staff client at stovoydbupg.wisconsin.edu address for testing upgraded
- Fix newbooks (tentatively completed 9:45am, Wednesday)

Switch over (in progress- starting at 10:30am Wednesday, completed at 10:50am)
- Take down read-only production zones (completed)
- Reboot upgraded zones to come back up as production
-- take down read-only production zones (outage starts here)
-- for upgraded zones -
-- zone configuration files back into place
-- /etc/inet/hosts in each zone back to production values
-- /oracle/app/oracle/product/10.2.0/db_1/network/admin/listener.ora backup into place
-- apache configuration changes if anything has an IP address hard coded
-- shut down zones
-- boot zones (should come up as production)
-- uncomment cron jobs for voyager and nightly backups
-- update nightly backup scripts as needed
-- on z3950 update z3950svr.ini and xxxdb/etc/ascopac/opac.ini and stop/start voyager (z3950.wisconsin.edu brief downtime at this point)
- Do any cleanup needed


No user avatar
bfwilson
Latest page update: made by bfwilson , Jul 28 2010, 12:04 PM EDT (about this update About This Update bfwilson Edited by bfwilson

103 words added
7 words deleted

view changes

- complete history)
Keyword tags: None
More Info: links to this page
There are no threads for this page.  Be the first to start a new thread.