Archive for July, 2005

Messing with IBM xSeries 336 1U and IBM in general, my experiences so far…

Tuesday, July 26th, 2005

IBM xSeries 336
Pictures @ eWeek

So in my current job I get to play with some cool toys, like the IBM xSeries 336 model 8837, and the EXP 400 Disk cabinet. I must say its the first time I have had some real hands-on experience with server hardware, and then to get to work with stuff thats pretty new is quite cool.

Add to that, the fact that its my job to get to know this stuff very well, something I am not accustomed to, is very cool. My past jobs where all pretty pretty boring compared to this, so very rarely was I able to play with rack-mounted servers, let alone new ones.

The company I am with is stardardising on IBM, and these servers represent the first IBM’s this department has laid their hands on.  The first 336 will be used as a WSUS server, which I have written de deployment plan for aswell. The two others we unpacked today, will be used in a cluster solution together with EXP 400 cabinet running a few different RAID configurations. These systems will be running a large logistics application that we dont know that much about yet, even though we, or rather, my female collegue, will be installing and setting up.

I am quite impressed with the hardware so far, but finding my way though the forrest of IBM’s BIOS, driver, and firmware updates, has been quite a hassle. To be blunt, IBM’s support site is a complete mess, with different versions of software included in different downloads, all currencurrently called the latest versions, while seperate downloads are almost impossible to find. Add to that a confusing naming convention, a very amateuristic download system, the most stupid website search function ever, confusing and contradictory documentation; its just been very tiring finding all the proper bits together to get these systems up to date once they are out of the box.

Here is a typical example

IBM offers a downloadable ISO on the IBM site that is suppose to autodetect and update ALL of your systems Firmware and Bios in a single CD-Boot cycle. Its called update express. But even in the newest version, it in fact contains firmware and bios versions that are outdated by single releases featured elsewhere on their support pages.

Oke, no biggy, we can always collect each update individually. Or can you? Most updates come as either a Zipped floppy drive image (the x336 doesnt have a floppy drive), or, thank god, an ISO. So getting our new X336’s up to spec, out of the box, requires at least 6 reboot cycles using 6 different bootable D-roms (!). Even installing IBM director was NO help.. nothing included at all that even hinted at any kind of automatic update system built in, let alone any system that could update firmware live, or, god forbid, remotely!

The only ray of light seems to be IBM directors built in software-distribution system, very very basic, but supporting a standard that IBM has to distribute driver updates (at least!), using a special package format. Pitty though that so far, only 2 of the 12 or so driver downloads for the 336 have this package format included in any way at all.

However, included on the UpdateXpress CD, is a little app called UpdateExpress Live.  This will, get this, automaticly download the updates you need for you. Does it scan your system first to determine what you need? No. Does it integrate with IBM director in any way? No. Does it even install anything? No. It just downloads them for you.
Not that it works mind you. The application contains a single hard-coded web-service URL that its looking for at the IBM site. And guess what.. its giving a timeout.. as in.. there is nothing awnsering on the IBM side.

So, just out of curiosity, I called the IBM support desk, got patched thought to Dubai or somewhere, and I actually ending up guiding the support guy though the steps of running the UpdateExpress Live app, just so he finnally understood what I was talking about! He obviously had no idea about how this was suppose to function. Then he proclaimed that they didnt support the UpdateExpress software, and proceded to guide me to … get this, the website feedback form! Its been a week and I have not recieved any kind of reply so far. I am hessitant to call again.

Here is another, really stupid example.

Try looking for the EXP-400 disk cabinet on the IBM site, then try google. Here’s another. IBM’s web-team have no clue.

Neither am I, initially at least, that impressed with IBM’s main system management software, the IBM Director itself. Sure it can read a lot of data sources and supports a lot of hardware alerting standards, but its interface is one of those examples of really really slow java programming. Its so slow to use the console, even on the server itself. But its web-interface is worse!  Clunky, slow, very limited, totally un-intuitive to use, and ugly.. its just not something I am very happy using.  All in all, it is powerfull though, and perhaps in time, as I learn to use more of its feature set, such as theRemote Deployment Manager, which may solve some of my above problems, providing we dont have to pay more.

So my experiences with IBM so far have been bumpy. Things will undoubtably get easier as I get to learn more how their miriad software and management systems work, and how to navigate their confusing web-presence.

Oh by the way, did I mention that lights-off, out-of-band management with IPMI is cool? Ever hearf of SAC?

Bought a Canon Powershot SD500 (IXUS 700)

Sunday, July 24th, 2005

PowerShot SD500

After my lodger bought a Sony camera, I was increasingly finding uses for it, and I started considdering in ernest getting one myself.

Read the DPreview of the SD500 here.

So far, I am extremely happy with it, and used it almost immediatly to shoot some pics of a party I went to.
The results can already be seen on Flickr, which I upgraded to a Pro account at the same time.

I need to spend some time learning how to use it properly though.

Windows Server 2003 Resource Kit Books

Sunday, July 24th, 2005

Yay! It arrived!

Gonna take a while to get though it. Once I do though, I promised Kevin a Admin-to-Admin book review

System Administrator Appreciation Day

Saturday, July 16th, 2005

Yes! Its that time of the yeah again! Mark 29th of July in your calanders

Time to do something special for the guy that keeps it all running!

Check out this fantastic sysadmin dedication song by Wes

Get your T-shirt here

ITT: SUS server Sync error caused by Surfcontrol and ISA (Solved)

Friday, July 15th, 2005

First week on the new job, problem nr1 solved.

(hostnames changed to generic names)

We where getting continuous Sync errors like follows:

Update for Windows Server 2003 (KB898792): Failed to download from URL ‘’. (Error 0x80072F78: Invalid server response.) – WindowsServer2003-KB898792-v2-x86-enu_ca965b3c805f48f2fe8ee1a420ddbf4.exe

Here is a copy paste of what happens on the ISA side: anonymous – N 2005-07-10 03:00:23 w3proxy SRV-ISA01 – – 80 – 150 2798 http TCP GET – – 407 anonymous – N 2005-07-10 03:00:23 w3proxy SRV-ISA01 – – 80 – – 738 http TCP GET – – 407 DOMAIN1 – Y 2005-07-10 03:00:32 w3proxy SRV-ISA01 – 80 9281 286 75921 http TCP GET application/octet-stream Inet 64 anonymous – N 2005-07-10 03:01:33 w3proxy SRV-ISA01 – – 80 – 147 421 http TCP HEAD – – 407 anonymous – N 2005-07-10 03:01:33 w3proxy SRV-ISA01 – – 80 – – 754 http TCP HEAD – – 407 DOMAIN1 – Y 2005-07-10 03:01:34 w3proxy SRV-ISA01 – 80 344 302 334 http TCP HEAD application/octet-stream Inet 200 anonymous – N 2005-07-10 03:02:20 w3proxy SRV-ISA01 – – 80 – 239 2798 http TCP GET…e1a420ddbf4.exe – – 407 anonymous – N 2005-07-10 03:02:20 w3proxy SRV-ISA01 – – 80 – – 827 http TCP GET…e1a420ddbf4.exe – – 407 DOMAIN1 – Y 2005-07-10 03:02:21 w3proxy SRV-ISA01 – 80 438 375 – http TCP GET…e1a420ddbf4.exe application/x-msdownload Inet 12210 anonymous – N 2005-07-10 03:03:07 w3proxy SRV-ISA01 – – 80 – 239 2798 http TCP GET…e1a420ddbf4.exe – – 407 anonymous – N 2005-07-10 03:03:07 w3proxy SRV-ISA01 – – 80 – – 827 http TCP GET…e1a420ddbf4.exe – – 407 DOMAIN1 – Y 2005-07-10 03:03:07 w3proxy SRV-ISA01 – 80 94 375 – http TCP GET…e1a420ddbf4.exe application/x-msdownload Inet 12210

Now the purely annonymous connections that result in the 407 are normal I assume, as you need an initial 407 from the proxy in order to initiate authentication information being sent from the client.

But after 2 x 407, ISA itself comes with its 12210 error, which MS states as: An Internet Server API (ISAPI) filter caused an error or terminated with an error

Well that is intereting!

Guess what we use on the ISA server aswell? Surfcontrol!
Surfcontrol is a web-content filter, used to stop people surfing porn or downloading crap.. its plugged into ISA server as a custom ASAPI filter.

We had the SUS server in there, attached to our “unrestriced internet’ rule (guess who else uses that rule? wink.gif

Anyway… the SUS server was defined in Surtcontrol by FQDN (Fully Qualified Domain Name).

It turns out, the way our ISA server is set up, is that its not doign any kind of resolving on incoming client requests.. the ISA logs show this; all clients identified by IP adress, no hostnames at all.

Because of this, Surfcontrol was unable to apply the rule, and was sending back 502 errors to SUS (Internally 12210)… which SUS translates as [b]Error 0x80072F78: Invalid server response[b]

Though changing the definition is Surfcontrol solved the issue, I am left wondering if this ISA behavior is by design or not. It might simply be the way we have it set up, I will have to look into this further.