Multihomed DNS and how Windows makes you lazy

I was called in to solve an interesting problem today. There's a good principle in effective IT security to segregate different services into LANs appropriate to their function. If there is more than one function, then more than one NIC (perhaps a VLAN ID) is required, with a unique IP address.

Windows is a great tool, a platform of continual development (and yes, that means it gets better, so don't think I've always found it to be great) over the last two decades that now runs a fair chunk of global business, and in some demanding environments. One of the simple beauties of the platform is the unified codebase, libraries and APIs from the smallest XP Embedded right up to Windows Server Datacentre Edition: The machine I'm developing on is almost identical, apart from scale, to the machine I'm likely to deploy on.

Yes, there are other differences, but I find most of them to be paid features like clustering and more speed (I can't do it captain). The hardware abstraction layer and other consistencies like the IP stack, filesystems and memory management are wonderful tools for developers. Unfortunately, so many admins cut their teeth on Windows desktop editions, or at least smaller servers under their absolute control, that they struggle to make the transition to enterprise administration.

My previous rant about NetBIOS is a case in point. With all this abstraction, details like network interfaces and network service location are so well hidden, they're essentially invisible. Ever try to catch the Invisible Man to ask him what he's doing?

The problem I had to solve today was around multihomed servers. Windows IT admins tend to be lazy, and NetBIOS broadcasts are only one factor where we rely on the wizardry of the OS to figure out what we're trying to do and make it happen. Dynamic DNS registration removes some of the tedium and mistakes from the process of getting systems deployed, but blindly assuming it knows what you want is just wrong.

The convergence of an Active Directory domain and the DNS namespace is a nifty feat, but in multihomed systems it's a nightmare. If all interfaces are routable and reachable, then this is slightly moot, but put up a firewall or routing restriction in the way and intermittent problems (the worst kind) crop up, and troubleshooting without a solid foundation in networking is tough. DHCP, DDNS, NetBIOS, even APIPA, all seek to hide the complexity from Windows admins, and they end up woefully underskilled in the cornerstone that makes their network tick.

The problem in this instance is that the FQDN of the server is comprised of the hostname and the AD DNS Name. No problem for a typical, single-NIC server. Unfortunately, this is an abstraction when it comes to multiple NICs: just how is DNS supposed to know what your topology is when giving you an answer.

Trying to convince a religiously Microsoft admin to use a subzone to specify the interface is absorbed with something approaching heresy. Do a traceroute (or tracert for Windows guys) to any internet address and you'll see FQDNs of routers, with the hostname portion wildly different along the way, including multiple digit groups. Most of these are Internet routers, and the DNS entries correspond to the interface rather than the router itself.

Of course, Windows has a mild cow if you try to refer to it by anthing other than the system name as the first part of the FQDN, and always expects all interfaces to be present in the machine's dns suffix. The best solution...

Change the way you think about finding servers. When you're connecting, you're probably interested in a particular interface anyway. Some services may not even be listening on particular interfaces. Getting your brain tuned to how your network is built, using that to figure out how your systems are connected, and habitually spelling out exactly which way you want to connect by an explicit FQDN can only do good.

Of course, some applications take it as read that a server is reachable by the short FQDN. Sometimes system admins can be even more hardcoded. Both are very, very difficult to change.

Putting an MCITP in its place

I have noticed that the new raft of credentials from Microsoft don’t necessarily make sense to folks, especially those that are already familiar with the old set of MCSE-type credentials. I mentioned to some friends that I've got the “new MCSE”, a lot of them got it, but it dawned on me that this is, in fact, a field of some confusion. A quick search on Google came up with one fault, and that is how this new credential relates to the ones we (certainly I) already know.

The point of any credential, be it Cisco, VMware, Microsoft or embroidery is to show to an external party that you are qualified in a particular field of endeavour. This was quite plain with Microsoft’s old regime, the Microsoft Certified Professional (MCP), and the Microsoft Certified Systems Engineer (MCSE), as well as the Microsoft Certified Database Administrator (MCDBA). However, as Microsoft branch out into new fields and offer solid, integrated products in fields not entirely related to Windows Server or SQL Server, the approach of cobbling together a new acronym for a new product or role is unwieldy – imagine the Microsoft Certified System Centre Engineer – MCSCE??

So, what is the transition?

MCP –> MCTS

The first Microsoft certification I got in 1997 was an MCP: Windows 95. This showed that, according to Microsoft, I was competent in installing, administering and troubleshooting Windows 95. I remember just how proud I was that day.

The problem though is that the term, “Certified Professional”, encompasses both the specific credential and the entire field of Microsoft certified persons, so is not entirely appropriate. “Technology Specialist” on the other hand, clearly shows what the candidate is trying to demonstrate, that he knows his stuff on a particular product. This bit is key, a specialist in SQL Server configuration is not necessarily a specialist in database development or administration, and in larger organisations the roles are very clearly separate. The MCTS credential clearly segregates say, an application server specialist who can administer web applications, from the server network specialist who will hook it up to the various internal and external parties accessing it.

MCDST/MCSA/MCSE/MCDBA –> MCITP

A big failing of the old MCSE credential was the elective system. While Microsoft may introduce the idea in the future, I sincerely hope not as it adds doubt and confusion to the mix.

I hold an MCSE on Windows NT 4 (incorporating the MCP on Windows 95 I mentioned above). It included two “elective” exams from a list of many more, specifically TCP/IP Networking and Exchange Server 5.5. This means I need to explain to anyone asking just what kind of MCSE I’ve achieved. This was partially remedied in the 2003 track to include an MCSE: Messaging credential, but no such moniker exists for a SQL Server specialist.

The phrase “Systems Engineer” was especially limiting, since it implies an ability to design and implement server infrastructure centred on Windows Server. While that is indeed my own focus, it is of little use to someone specialising in monitoring and management systems, or even the venerable desktop support guru. While the DBA and the Desktop Support guy had their own acronym (MCDBA and MCDST respectively), I certainly don’t want to have to memorise the ever-growing list as a hiring or support manager.

By asserting that someone is an IT Professional in a named field, it indicates a proficiency in a technology set rather than one product. It also narrows the competency; while an Enterprise Administrator demonstrates competency in designing and implementing infrastructure from SANs and Terminal Services down to the desktop, the Server Administrator credential is more focused on those with competency in Windows Server itself.

These credentials are not easy to come by, and are especially hard if the individual has no relevant experience in the real world.

While the plethora of MCITP credentials may seem like a dilution of the fairly focused MCSE, it offers the opportunity for many more product specialist to demonstrate their competency in their field, with a credential on a par with the more established Systems Engineer we’ve come to know.

MCM/MCA

Now we get to the good stuff. The Microsoft Certified Master and Architect credentials are not for the faint of heart or newbies. The intensive certifications are for those with five or more years experience leading complex design, implementation and migration projects and a demonstrated history as a technology leader and expert. Standing up in front of a panel of recognised experts purporting to know your stuff is a daunting proposition, probably even for a few of the members of the very panel you’d be standing before.

For anyone claiming to be hot stuff on the range of Microsoft products, services and solutions, this is where you should be aiming. If you’re already that good, convincing your company to stump up for the three weeks training in Redmond for the MCM should be no effort, and I look forward to getting to that level myself.