Service desk requirements.

July 28th, 2010

This’ll be expanded upon with some explanations.

Site license! – there are more users for this than you expect. (Note, I’m not ruling out open source here.)
Email from “Everybody” – no preregistration. Eg, HPC, bb-help.
Configuration and integration: extensible API, no proprietary little languages (where are my interface libraries to talk to MQ, etc, in the little language, if needed?)
Clever UI widgets.

Using RT to expose, measure, and improve our performance

May 29th, 2010

I keep on talking about “tool support” as being a vital part of any approach that attempts to address the problems of the department and I think it’s worthwhile spelling out what I’m going on about. I’m going to try to offer a concrete illustration of how we might use a software tool to help us with a number of the things I keep on bashing on about.

I use RT as the concrete example of a ticketing system, but if you are (rationally or irrationally) turned off it for some reason, replace it wih ANY lightweight ticketing tool that supports a rich set of inter-ticket relationships.

I’ll start by listing some of the problems the department has (if we didn’t have problems, there would be no need for an organisational rethink), then describe how one might utilise a ticketing system to help, and finally outline the benefits.
Read the rest of this entry »

Oracle calendar to iPhone external calendar (.ics format)

October 28th, 2009

More here:

This represents a bit of a rejigging of the to give Events the ability to turn themselves into iCalendar-style records. They also pick up vAlarms.
Read the rest of this entry »

Solaris cluster, MPxIO, Zpools.

June 26th, 2009

This is remarkably straightforward.

Two nodes, alike in dignity. We preconfigured MPxIO on them prior to installing the cluster software. We also allocated a small LUN that was visible to both nodes prior to the installation, and made sure they could see it: this was intended to act as a quorum device.
Read the rest of this entry »

Zenoss (general Zope) behind an Apache proxy.

May 28th, 2009

Zenoss’ web server lives on Zope. This doesn’t sit perfectly well behind apache because we’ve configured it to listen to localhost:8080; that is the address it will default to stashing in its response pages.
Read the rest of this entry »

Braindump of FC multipathing on Linux

March 27th, 2009

Start with the cards:

# /sbin/lspci

04:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
04:00.1 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
05:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
05:00.1 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)

Two dual-port cards, effectively.

Kernel module for these:

# /sbin/lsmod | grep ql
qla2xxx 1079969 0
scsi_transport_fc 73673 1 qla2xxx
scsi_mod 188665 7

qla2xxx, already loaded.

To find WWNs, on RHEL 5.x (centos 5.x),

WWN information and other FC stuff is under
for values of N.

# ls /sys/class/scsi_host/host*/device/fc_host*/port_name

(there’s a host0 on this box which is the onboard SAS controller)

# cat /sys/class/scsi_host/host*/device/fc_host*/port_name

So these are the addresses that need zoning.

Scanning for LUNs:

# echo "- - -" > /sys/class/scsi_host/host1/scan
[wait a bit]
# more /proc/scsi/scsi

[new paths to luns show up, mpaths appear under /dev/mapper if that’s configured]

After scanning for luns (so they show up in /proc/scsi/scsi)…

/sbin/chkconfig multipathd on
– edit /etc/multipath.conf to look like the one on bastet. I made the following changes:

blacklist {
devnode "*"
blacklist_exceptions {
devnode "^sd[b-z].*"

(since sda was always the local SAS root device) and modified the defaults {} section to use “failover” rather than “multipath”, “multibus” etc, which appears to be fine.

After you’ve done this you should be able to do the following:

/sbin/multipath -v2 -d

-d is for dry-run (make no changes). It’ll tell you that it’ll make mpath0, mpath1, mpathxxx, tell you the ID of that volume (a long hex string) and what paths are available to it, what SCSI vectors those use, etc.

You can then add a multipath{} section to /etc/multipath.conf which lists that WWID and gives it an alias (ie, doesn’t use “mpath0” etc) – we used “sb2cc-lun5” as an example alias.

Start multipathd if it’s not running: /etc/init.d/multipathd start and (with a possible /sbin/multipath -v2 to search for new paths and make them available or to rescan multipath.conf) you’ll find /dev/mapper/sb2cc-lun5, etc, as new block devices.

We labelled these with e2label, made an ext3 filesystem (takes several minutes for a multi-TB filesystem) on there and put entries into /etc/fstab as follows:

/dev/mapper/sb2cc-lun5 /sb2cc-lun5 ext3 defaults 1 3

You probably don’t want to use ext3 for a large filesystem.

These come back happily AND IN A STABLE FASHION on a reboot, which is just as well because the raw “path” devices, sdb..sdi were juggled on the reboot – this is just down to how fast the various HBA scans come back.

– Listing current multipath settings: multipath -v2 -l
– Listing what would be changed (no changes made, dry run): multipath -v2 -d
– Making those changes live: multipath -v2

Getting rind of unwanted LUNs:

# /sbin/multipath -v3 -d
[nothing changes, need to rescan]
# echo "- - -" > /sys/class/scsi_host/host1/scan
[wait a bit]
# more /proc/scsi/scsi

[new paths to luns show up, mpaths appear under /dev/mapper

At that point I noticed that the backup host group on sb3cc has access to some stuff it shouldn’t; turned that off. Now need to flush the paths to the LUNs that’ve disappeared:

# /sbin/multipath -F
# /sbin/multipath -v2 -l

only two paths show, scan for the LUNs on the other controller…
# echo "- - -" > /sys/class/scsi_host/host2/scan

Now you can create the multipath devices.
# /sbin/multipath -v2
# /sbin/multipath -v2 -l

two large LUNs show up…

mpath6 (36000402001fc475761ee919c00000000) dm-5 NEXSAN,SATABeast

^^^ this is sb3mvb (…4757 in WWID)

mpath5 (36000402001fc46db60ef903200000000) dm-2 NEXSAN,SATABeast

^^^ this is SB3cc (….46db in WWID)

At that point you I edit /etc/multipath.conf to give these useful device names, and run /sbin/multipath -v2.

Kerberos patch finally in the JRE.

December 11th, 2008

Using oracle_calendar from within Zope.

August 15th, 2008

Zope (2) wraps a proxy around objects retured from an External Method (or behaves in a similar way) that protects member access from other python scripts.

This is exactly not what I’m after. Apparently this is fixable by adding

__allow_access_to_unprotected_subobjects__ = 1

to the class definition. It’s irritating to have to modify external methods in order to make them Zope-ready; at least the fix is (apparently) a one-liner.

oracle_calendar in python.

August 13th, 2008

There are several blog posts out there decrying the woeful state of SOAP in the Python world. I shall not echo these in detail; suffice to say, they’re absolutely right.

Mind you, there is also a bunch of lack in the Oracle calendar SOAP interface: no WSDL, no querying of delegate calendars, a limited SOAP implementation that appears to have been written in terms of SAX events and that’s particularly sensitive to namespace declarations, and so on.

Anyway: I got enough of this working that I can query a user’s calendar in a fairly Pythonic fashion – for details. It’ll suffice to reimplement the guts of Shirley, the ILRT’s automatic receptionist.

This also marked the first time I’ve tried using new-style classes to augment the Python str class with a secondary attribute.

Having tested this tonight, we can indeed pull out a list of ilrt-visitor meetings, together with their organiser’s details (mailto and cn).

On the production status of the Departmental Filestore

June 10th, 2008

The Google TechTalk on the subject of Scrum, given by Ken Schwaber, contains one of my favourite quotes. You can see the whole thing here; and if you haven’t, it’s worthwhile devoting an hour to watching it. Can’t be bothered? Then don’t bother reading on. And the quote? To paraphrase,

our discipline has a tried and tested way of going faster. Cut corners, cut quality. That way you can produce more crap.

So, how does this relate to the DFS?
Read the rest of this entry »