Deployment work, up and coming.
I’ve just physically plumbed in to the deployment & management network two new boxes, spam3 & spam4. These will be targeted by an MBS instance for deploying SpamAssassin (& co) configurations onto.
There’s finally a light at the end of the tunnel with the filestore work. Once the HBAs turn up I’ll be putting a new NFS server in place. This impacts on the appinf work in two major ways:
1. Cool toys. NFS as an environmental service from farm hosts. There are some other tomcat deployments knocking around that require persistent storage. I’d like to build these into the mix so that we can offer a robust platform for the wikis, etc, that are knocking around; an NFS platform is also a requirement for work such as any work targeting the blackboard application tier at the web farm.
2. A change in the way the MBS databases are served up.
There was a tentative plan to SAN-attach the MBS. I’d much rather supply its storage via NFS, which is must faster to find a replacement host for. My plan is to rebuild the MBS setup on hereford (currently a farm host
on PRE) and punt defiant back into the farm.
We’ve seen some issues in keeping a pair of MBS DB instances in sync; I’m looking forward to using some of my upcoming copious free time to do some fairly hefty development work on the MBS side (which is long
overdue). In order to make this work smoothly and to help with patch management, I’m planning on making a change to the way the MBS is laid out, as follows:
- all MBS checkouts will go directly onto NFS-supplied disk. We’ll also be supplying home directories from the NFS cluster so that we aren’t in any danger of running out of elbow-room.
- the mbs/src, mbs/zfs, mbs/apache and other “common infrastructure” parts will live under one repo, laid out as follows:
mbs.src/trunk HEAD, development happens here (or on dev branches)
mbs.src/branches/mbs.pre HEAD pushed out to here
mbs.src/branches/mbs.prod …and here
mbs.src/branches/... …and so on.
The mbs.src repo will not only contain the code for deployment methods; it’ll also have a skeletal mbs/db that contains the core artifacts common to all database instances: the platform-deployment, bootloader, and os images. These can then be maintained from a single spot.
All merging out to live branches will be under the auspices of svnmerge, or svn merge if 1.5.0 is out by then.
- all MBS database instances will then be built as separate repos. Each repo will contain svn:externals pointers to the src/, db/image, etc, subdirectories that should be centrally maintained. I may well punt software images (apache, tomcat, etc) into this too.
- This shouldn’t make the blindest bit of difference to any of the scripts, with the exception that the apache users.xml and groups.xml will need to be moved out of those particular subdirectories.
- checkouts: finally - I’m strongly tempted to run each MBS instance under its own “zone” using OpenVZ. There have been some issues with the PXE platform-deployment instances getting in the way of each other; this’ll let me split them up cleanly.
Schedule: once the NFS service is bedded down I’m going to begin this work; expect it to run over the summer. At some point there’ll be a cutover as the new MBS comes into service.