Archive for the ‘IT Policy’ Category

Phasor Burn: The Official Unofficial System Administrator Oath

Friday, April 10th, 2009

This is basicly a report of something the Phasor Burn blog linked to.

Very nice blog btw, and he linked to my related rant on there, thank you for that 🙂 The original is from an alt.sysadmin.recovery post from ’99 by Mike Sphar, and it deserves reprinting:

I am hired because I know what I am doing, not because I will do whatever I am told is a good idea.

This might cost me bonuses, raises, promotions, and may even label me as “undesirable” by places I don’t want to work at anyway, but I don’t care.

I will not compromise my own principles and judgement without putting up a fight.

Of course, I won’t always win, and I will sometimes be forced to do things I don’t agree with. My objections will be made known.

If I am shown to be right and problems later develop, I will shout “I told you so!” repeatedly, laugh hysterically, and do a small dance or jig as appropriate to my heritage.

I am not sure its in our own best interest to actually show we feel rather self-satisfied at times when things do go wrong, but we all feel it of course. Sadly, within the complexities of corporate politics, its sometimes quite hard to show clearly how your the disregard of your advise is directly to blame for the issue at hand. But when our moment comes, savor it, but savor it privately 😉

First Impressions of new job

Monday, April 6th, 2009

Today is my fourth day at the new job.

Office3

In some ways I feel I have taken a step back perhaps. It feels very much like many of the companies I worked for before DHL. Medium-to-small IT departments, medium-to-small user base. A lot of overlap between the roles of helpdesk (that they called the “frontoffice” here), and systems administration.
They department consists of 8 permament staff, including me, and 3 temporary guys. The department head of IT, 3 application administrators, 2 part-time people in the servicedesk, and 2 systems administrators (including me), including myself.  The team supports about 120 users, most of them at this location. The temporary guys are here to help with the extra workload they are currently suffering, and help out with both servicedesk and sysadmin stuff.

Office4

Office5

The role-devision between the servicedesk and the sysadmin team is not clear at all, meaning that a lot of stuff is being loaded on to me and my collegue. We also share the function as second-line desktop support, meaning we support the users directly  with their enviroment. This last part is not something I am particularly happy about, I had hoped to have left that kind of stuff behind me.
However, I do know that the best kind of IT support I could possibly give users is my own hands-on approach. I am confident in my ability to approach and talk to users, and I can usually figure out their story just by listening. I speak clearly, I act professionally, I am good at this. However, I dont like interacting with users. But you know, and this is what I am telling myself right now. 120 users is not a big group. Its a group of intelligent people, patent lawyers many of them. Its a far cry from the average warehouse employee at DHL. Right now its pretty hectic, and a lot of relatively simple second lien work is being piled into my collegue, the only other sysadmin. He is very gratefull I am here, and I have already taken on some responsibilities to ease his burden.
I expect therefore that once there is a bit of order in the chaos, and I get the chance to work with the more interesting stuff, I wont mind that second-line stuff as much.

The environment they are running is interesting.
– 120 clients, many thin clients.
– 4 VMWare ESX servers running about 30 Application and Database servers
– About 20 other production servers with 7 Citrix Metaframe 4.5 servers that provide users with a standard desktop
– RES Powerfuse as a scripting layer on top of Citrix
– RES Wisdom as a client management tool for installations, etc.
– Microsoft Softgrid for application streaming to the Citrix servers (keeping them clean) and to some clients.
– HP SAN
– HP Blades
– The usual infrastructure stuff like a Windows domain, WSUS, HPSIM, Windows Deployment Server, etc.

Of the above Enterprise technologies, the only one I am really familiar with is RES powerfuse and the infrastructure side. All the other specific third-party stuff I am either completely unfamiliar with, or familiar with older versions.

The enviroment is brand new, having been designed and set up for them by a consulting company, and there are many many loose ends that still need tying up, mostly from a procedural standpoint.
This is specifically why I was hired. Its obvious to me I need to take a lead role in laying out and implementing some standards, as they have more or less none at the moment.

To this end, I have already discussed using Microsoft Sharepoint as a department collaboration tool, starting with nothing more complicated that change logging. So I will soon be setting up a server for this, I rolled out a new virtual server this afternoon.

Office

Office2

Office1

In the meantime though, my job is first and foremost to take some presure of the other sysadmin. I have been given a list of long-time, but asyet unsolved common problems users are facing, most of them related to the Citrix desktop. I will be analysing this list tomorrow and seeing if there are any quick wins amungst them.

Its hard not to feel overwhelmed by the barrage of work that these guys are getting piled on top of them. my collegue has to spend most of his time on a number of important projects that are currently being rolled out, this while all this other stuff is still lying around. I have to remind myself to work at my own pace, and not get caught up in the near-panic around me.

A more simple quick win today, was that I took it apon myself to clear out the server/network rooms, that where clogged with boxes and other junk. Me and the facilities manager spent the afternoon clearing about a dozen boxes of junk, and I managed to sqeeze what was left into the storage closet.

This may sound like a silly job, but it was really nagging on my collegues mind. It was just one of dozens of other things he cannot get round to sorting out. After he showed me what could go and what to keep, I just told him to relax and that I would take care of it. He looked incredibly relieved. I think he has really missed the ability to be able to offload some responsibility onto someone else.

This kind of thing also makes a good impression on the manager of course. I am not explicitly setting out to impress, but hey, it cant hurt can it?

Here are some more pics I made this afternoon. They are a little crappy but I will promise to make many high quality pics as usual, eventually. My Cano G9 is a little harder to hide than my G1 phone of course 😉

Serverroom1

Serverroom3

Serverroom4

Serverroom2

Serverroom5

I have create a new set on Flickr, called “Current Work”, where you will be able to find all these pictures and many more to come

Desicionmaking on the new Proxy solution

Wednesday, May 28th, 2008

For your enjoyment, here is a, slightly edited, email I just sent to the department head and various other decision makers. It goes over some of the options we need to consider to solve the current issues with our internet access.

Names and places have been changed to protect the guilty 😉

And please exuse the spelling. I was in a hurry and I really dont care about spelling as much as I do content.

—————————-

Hi all,

We are currently faced with some decisions that need to me made in regard to the Internet Proxy solution for the Netherlands and Belgium.

This is the current situation in regard to the proxy servers in Lala City and Chipville.

Server Lala City: LA-Server-S99
Server Chipville: CHIPVILLE-Server-S99

Both servers are HP DL360 G2 servers, and are now approaching 8 years of age. They are very out of warranty, and no hardware support can be expected from HP anymore regarding these.
Both servers run Windows 2000 standard
The Proxy software on both servers is ISA server 2000, running on the SQL MSDE engine. This software is still supported by Microsoft, but has been superceded by 2 newer versions.
In addition, we currently run the Surfcontrol web-filtering software, as a plug-in for ISA.
This software allows us to tightly control web-behaviour, for example to allow certain users access to certain sites, and to block entire catagories of websites, or web-protocols.
We have built up a pretty extensive rule-set over the years on both machines, and both rulesets are largely identical.
The company “Surfcontrol” was aquired by Websense in 2007, and since that time the Surfcontrol software is no longer supported, no patches or service packs are being offered for download, and no licences are being extended or sold, forcing all former Surfcontrol customers, including us, to look for alternatives.
The software combination on these servers has causes us some issues in the past. Some elements of Surfcontrol have always been buggy, and as the hardware has aged, it has become unreliable.
Furthermore, the decision to use SQL MSDE has causes problems, because of its inherrent 2gb limit.

Lala City
The Lala City proxy server is due to be replaced with new hardware, located in SiteB. This action is outstanding as part of the TCR move project.
As part of this, a new server was purchased, together with W ISA server 2006, and SQL 2005
At the moment, no replacement for Surfcontrol has yet been purchased, although Dick Dickerson did get a cost estimate for the Websense software, based on a single server, 500 users, and 3 years of licensing. (included as attachement)
A decision on this has been on the back burner, due to the fact that we where also planning on moving the current ISA server to SiteB anyway, and using the Chipville ISA server as a backup.

Chipville
The old Proxy server in Chipville is in a similair state to the one in Lala City. Although one of its 2 disks (that run in a mirror) has failed since last week.
This causes a serious risk to internet service continuity. It also represents a risk to the TCR move project, as this server is now no longer a reliable fallback while we move the Lala City server.

We need to decide how to proceed going forward.

The time factor
We have only a limited time to come up with a solution. Currently the situation in Chipville is more pressing, because of the hardware failure of the server there.
The big-bang server move from TCR Lala City is sqeduled less than a month from now, and we need a stable and supportablesolution at the very least in Chipville before that time, and idealy a solution for Lala City aswell.

There are a number of options:

Option 1. Keep the current servers
The Lala City server can be moved to SiteB and continue to operate from there, serving Internet users (non-citrix) in the Netherlands.
However, the hardware and software is no longer supported, the software is in an unstable state due to past problems with Surfcontrol, and the ISA MSDE database.
Due to the advanced age of the hardware, it is only a matter of time before it fails. Moving it might actually break it too.
The Chipville server cannot operate as-is, on a failed hardware mirror. This absolutely needs to be replaced, more or less disqualifying this option.
Due to the above, I cannot recommend this option in any way.

Option 2. Outsource the Proxy service to European Datacenter / UK
This would involve redirecting all internet traffic from Netherlands and Belgium to an outside, centralised Proxy system for internet access.
This would simplify our support model somewhat, and remove the technical burden of supporting the solution ourselves.
The downside though, is that we no longer have direct control over what is allowed/disallowed over the Internet.
By default, as far as I have heard, no rules are in place for both the UK and European Datacenter proxy solutions, meaning that there are no limits on what people can do with the Internet connection, it would be a free-for-all, whereas right now, we have strict limits on usage.
This option should be considdered. But the question has to be asked why the web-filtering function was ever needed in the first place. If web-filtering and control remains a business requirement, that this options cannot be considdered.

Option 3. Hybrid In-country hosting / European Datacenter hosting
I have been made aware of a version of the European Datacenter hosting scenario, that includes re-directing in-country internet traffic to European Datacenter, but in combination with a local Proxy/web-filter server, running the Websense software. This would involve installing a local server with the “Websense” filtering software, and “chaining” it to the Websense Proxy server in European Datacenter. Many countries apparently already follow this model.
This has the advantage of retaining local control of a rulebase, allowing us to continue to restrict internet use where nessesary, but with the advantage of not needing local Internet line for basic Internet use anymore. MEGACORP(TM) also can retain an amount of corperate internet-use control, via the gateway in European Datacenter, as all internet fraffic eventually moves through there to get out. Currently MEGACORP(TM) does not pose any global restrictions on the Internet gateway in European Datacenter, as far as I have heard.
This option should be considdered, however it will take some time to study and set up properly. The support model may be complicated because of the fact you are dealing with possible web-filtering and proxying in 2 different locations, supported by 2 different organisations. It would however, also require that local websense software be purchased and supported. I have also been told by some, that the connection via European Datacenter is very slow and not that usefull for many operational tasks.  This could hurt us, as we run a number of line-of-business web-based applications over the internet. (Hp Shipview, etc)
We would also benefit from the fact that the websense software can be centerally managed from 1 console, making it very easy keep the netherlands and Belgium ruleset identical, and simplifying reporting and failover.
I would recommend this option if we can be sure the performance is adequate for our business needs, and if the support model can be agreed apon quickly. The major downside of this solution currently is that it will take time to set up, and we dont have much time anymore.

Option 4. New installation In-Country
This involves basicly rebuilding the 2 Proxy servers on new hardware, and installing fresh, current and supported Proxy and web-filtering software.
In this scenario we would use our own local Intranet lines in SiteB and Chipville.
We would directly support the solution, and maintain direct control over the web-filter ruleset, this is the most simple support scenario.
Hardware for this is already available: The replacement of the Lala City server was already part of the TCR move project, as is the licence for ISA 2006 and SQL 2005.
Hardware for Chipville is also already available on site, in the form of a 3-year-old IBM server, however, this server may soon fall out of hardware support (needs to be checked).
Apart from the ISA and SQL license that would be needed for the Chipville server, we need new web-filtering software for both servers, again, IF the business still deems this a requirement.
If they dont, then this solution would provide unfiltered internet access to all (non-citrix) internet users in Netherlands and Belgium.
For the web-filtering requirement, I would at this time advise to als go with the Websesne software, as they are currently regarded as the market leader, and their software is well supported ans well known in the industry. (they are incorperating a lot of the Surftcontrol concepts as part of the aquesition )
We need to look at the current available hardware for this. Almost all the hardware we have is 3 years old or older, so it may be advisable to considder purchasing a new piece of hardware for this solution in Chipville.
This option should be considdered. It has the advantage of retaining central control and will be quick to set up, once the software has been purchased. The downside is that the Websense software is expensive, so we may want to considder looking at alternatives, even though it has becomes a defacto standard within MEGACORP(TM). Again, we have a time-constraint problem here.
We would also benefit from the fact that the websense software can be centerally managed from 1 console, making it very easy keep the netherlands and Belgium ruleset identical, and simplifying reporting and failover.
I would recommend this option first and foremost, and it is the prefered solution technically, considdering the circumstances.

Again, i wish to stress the timeconstraints we have, less than a month before big-bang, we want a new solution up and running within the next 3 weeks!

Security Park – It is the employer not the employee who is the weakest link in a companies IT security

Tuesday, June 21st, 2005

Security Park – It is the employer not the employee who is the weakest link in a company###s IT security

It is the employer not the employee who is the weakest link in a company’s IT security

SurfControl has today announced the results of a new UK survey that uncovers an alarming level of complacency by employers when it comes to combating spyware in the workplace. The poll found that 21.3 percent of all respondents’ employers did not prohibit the use of Instant Messaging to contact friends, Web-based email, recreational surfing, downloading free software, personal online banking, storing personal files, sharing free music/video files, playing online games, running CD-Rom/DVD media or the use of USB flash drives on work PCs.

Read more

—————————————-

Well I agree with the sentiment.. well some of the sentiment, if not all of what they claim are ‘threats’.

One has to bare in mind who sponsored this report, and who is presenting the news: Surfcontrol; and they have a rather large stake in this kind of discussion.

Litterally anything can be a threat if you look hard enough. I would not call IM-ing friends a threat. I might call file-transfer via IM a threat, but not much of one…    Use of USB drives?  Well its the same issue: not being able to fully control what files pass in and out of your network.

At the moment, with the current state of affairs when it comes to files and file-systems, I would say its just about impossible to lock down your network to stop foreign files from entering your network. They trick is to mitigate what threat they do pose. AV on the desktop is one part of that, a strickt and enforced lockdown policy of the desktop enviroment is another..  and the same can be said for permiter defenses…

Its that old cost vs usability vs security arguement. You can have a little of all three, but not all at the max level. People use IM and play games to give themselves a little distraction, which I believe is a healthy thing, in moderation. Not to mention IM being the perfect productivity tools if used for work purposes.

USB sticks? Well they have taken the place of floppies. I often see people resulting to USB sticks if its the easiest alternative for getting to their data. Shutting off access to USB may mitigate some of the foreign-file threat, but I dont think it stands in relation to the added support costs you incurr, or the effect it has on worker morall. Instead, perhaps you should be focussing on giving your users what they need: Easy (and secure) access to their files; remove their reason for trying to work around the system.

And what the hell is wrong with ‘Web-based email’, ‘recreational surfing, personal online banking’. How is this a security threat? yah sure.. downloading trojans perhaps .. spyware? Mabe.. .  .. how about a software restriction policy then? If you run windows 2000 and up, you already have the mechnism to impliment it…  just a case of doing it.
How about locking down Internet Explorer? Turn of ActiveX via group policy.. its not perfect.. but its a start! Think about running Firefox on desktops yet…  might be worth considdering!

I am against the view that Surfcontrol seems to take, that any freedom you give employees, both online and off, is always a bad thing.  Try turning off all net access in your company, and lets see what it does for morale? Work should be a place you want to go to, or at the very least, not mind going to, so that means employees should be giving at least some thought to distraction and relaxation, finding that balance of productivity and fun. Blanket blocks on certain activities are not the awnser, a far more nuanced approach is needed that combines and weighs out those important ellements in the way that best suits your companies needs: cost vs usability vs security.

Symantec Internet Security Threat Report Volume VII

Monday, March 21st, 2005

Its not been posted just yet, but various sites are reporting the release of Symantec’s Internet Security Threat Report Volume VII.

Here is a link to where it probably will be posted:
http://enterprisesecurity.symantec.com/content.cfm?articleid=1539

With this report, keep 2 things in mind:
1. Symantec has a vested interest in selling and marketing their own Antivirus/spam/spyware products.
2. Symantec is nontheless in a good position to compile statistical information on threats detected.

Reports like these are, in my experience, treaded in one of two ways by IT managers. Or they are dismissed out of hand, for reason no.1 … or they are a more trustworthy source for knowledge on these matter than myself, partly for reason two, but more cause its an ‘official report’ .. and managers often dig that kinda stuff..       Yes.. im my experience many IT managers are that shallow.

Anyway.. I will have a page thought it later on.