Streetview and WiFi - Courts Need Some Education

I'm hanging my head in my palm in a manner not unlike the Jean-Luc Picard meme today, after reading a decision by a court in the United States. EFF has a great article summarising the effects, but I'd like to expand and go into the cause too.

So Google drove around with an antenna on their Street View cars for a few years, sniffing for wireless networks. This is very useful if you'd like to know your location but can't get a GPS signal, especially on devices with lower quality antennas or a location with poor sky visibility. As long as you have a data connection, you send the local Access Points off to Google's servers, and they will look up the location and feed it back to you. Simple, right?

Well, sniffing for access points (AP) is ridiculously simple. Your phone does that every time you look at a list of networks in your location, by looking for a special network frame called a beacon, which a regular access point send out roughly ten times a second. It's crucial to how WiFi works. It uses the same frame type as normal traffic, so even without a beacon you can still see at least the presence of traffic, and the MAC address associated with the AP. If the network is unencrypted, your WiFi network card automatically accepts those traffic frames too, then has it discarded by your kernel because it is not destined for your computer/smartphone/tablet etc ("your device").

I've done sniffing myself, in a practise known as "war driving". It sounds ominous, but it's also a very interesting excercise, for which I purchased a specific Atheros WiFi NIC, thanks to their products having excellent Linux support. Hook up a GPS receiver, and just go places. The software figures out where you are, looks at the list of APs nearby and pins them to a map. The problem here is that simply enabling your card to listen for APs does cause your system to store those traffic frames, since they are usefuol for determining the IP range in use on that network. Note I'm not trying to use those networks, I'm just interested in seeing how they're being used.

Now comes the interesting part: The court decided these signals being intercepted were not "radio communications" (despite being carried by photons) for the purposes of a legal interpretation, not "readily accessible to the general public" without "sophisticated hardware and software", and finally "most of the general public lacks the expertise to intercept and decode payload data transmitted over a Wi-Fi network".

On each point:

• It's radio. Learn physics. The lower court's opinion that the law covered "predominantly auditory broadcast" is an inference form the court; 18 USC § 2511 mentions audio only for satellite transmission, and only then to describe an audio channel used as a carrier for digital communications therein. If I speak ones and zeros into my walkie-talkie, does this suddenly become digital communications? Your meaning is divorced from reality, and I think it suits an agenda instead of fact.
• If I walk down a street, glancing into shops as I go, and see a person in one shop/office/etc handing a big photo with a red X through it to another person, with person in photo turning up dead and either of the two persons implicated at trial, I can't be prohibited for testifying because I got information not "readily accessible to the general public"(i.e. they were not standing on a street in plain view); a police officer seeing the same thing under the same circumstances is not prohibited from using the information for lack of a warrant or probable cause - it still doesn't affect the fact that photons bounced off the photo and got interpreted by "specialised hardware " (eyes) and "specialised software" (brain). Nobody's privacy is invaded, but information was blasted into the street nonetheless.
• WiFi NIC cost: $10 (shipping costs vary), down to as low as one. These devices are sophisticated, don't get me wrong, but then most computers are mind-numbingly powerful today. Linux kernel cost: nothing (download costs may vary, but is unlikely to make you hit your ISP's download cap),  easily installed in an array of distributions. Laptop cost: Variable, but if you've got one lying around you already have your solution.
• Interception is easy stuff. Decoding is easy stuff, and your device "decodes" it as part of its' primary function.
• A further point the court asserted is that regular (AM/FM) radio communications can be received miles away, versus WiFi that "fail to travel far beyond the walls" of a location. Again, physics. Oh, and define "far" - my balcony is less than 15 meters from my AP and my signal comes and goes, while the street is 30 metres away (in the opposite direction) and I still get an association (but not much throughput) fairly reliably. I despise vagueness in court proceedings.

The fact is, sophisticated software is required to not receive the payload. If somebody decides to configure and use an unencrypted access point and I happen to walk past with my phone doing the searching (I left the WiFi on when I left my home), or researching which models of router or service provider is prevalent on the street, or finally if the NSA, FBI, federal or local officials are parked in a van across the road, simply turning on the function puts the traffic in RAM. Even if it destroyed a microsecond later, under this ruling I (and they) have broken the law. The bar for warrant-required searches just shot up.

I'm used to seeing courts being out of touch with reality, especially in computing cases, but this is beyond unreasonable. I have no doubt Google had zero intention of capturing user detail such as e-mails, usernames or passwords (why would they, they run an e-mail service?), and are now being prosecuted for users' inability to secure their own networks.

But mostly: Physics!

Locking myself out

I've become quite adept with Android ROMs of late. In the last month I've re-flashed my HTC Legend at least eight times, one of them a desperate measure to get back in after idiotic inattentiveness.
The after-market firmware scene is amazingly vibrant, and I'm typing this post on Cyanogenmod's CM7.2, which brings Gingerbread (Android 2.3) to my two-year old device, something the manufacturer has no intentions of doing. Just as in my previous post about digital legacy, the open-source community is doing what vendors simply won't, and surprisingly well. I'm so pleased I've even offered to upgrade a (decidedly untechnical) friend's Legend.
Doing this requires bypassing the vendor's OS locks - "rooting" - and optionally the firmware locks. By default, open ROMs leave these locks off to preserve your Newfoundland freedom. There's a problem there.
Yes, I want to continue hacking away, but I'm not comfortable leaving root accessible for my friend's device. A diagnostic mechanism called USB Debugging doesn't reset itself on reboots rendering the screen lock rather useless. The Cyanogenmod authors have stated root access will be off by default from version 9 (Android 4.0) onwards, but the Tom I tried on my device (so very close to useable) had no such measure.
Should a rooted and reflashed device be misplaced or stolen, the new owner could quite easily get in. Remote wipe and location apps can be disabled (Link2SD makes this trivial) and data compromised. As a network administrator I wouldn't let one of these devices anywhere near my messaging system, since the policy enforcement engines rely in unprivileged users to function.
I could of course re-applt the vendor recovery software to prevent the OS image being altered, and re-enable firmware controls to stop this being undermines, but this is all moot once the OS can do as it pleases.
Open-source software thrives on freedom, and I love that attitude, but ensuring a well-controlled network often works against that when viewed from certain angles. So, what path?
Security systems relying on obscurity of design have repeatedly been subverted, but how do you convince open-sourcers to build effective blockouts into their project?

Perhaps by helping them realise the real loss their moms and kids might experience when their unrelentingly open device is abused?

Utility as a Learning Tool

IT, as with other high-skilled vocations, requires a constant cycle of learning and certification if you are to attain, retain and prove your skills. Each practitioner I know has specific competencies or bias towards particular product sets, architectures and vendors, so naturally we keep up with the latest developments and new releases.

One of the recurring problem engineers have is getting to grips with the ins and outs of a new product. Supporting the infrastructure for large organisations requires a lot of time to explore the feature sets (especially as they contrast with the vendor’s stated features), their utility, recoverability, capacity etc etc. This is quite entertaining in itself; incremental product releases generally tend to build on the feature sets of earlier versions, and if the vendor is any good they’ll provide good documentation and training for the upgrade path.

The one aspect I have problems with is a brand new product set from a particular vendor. Headline products servicing databases, messaging and operating systems are very infrequently created from scratch, but the myriad supporting products and protocols are under constant evolution. Quite often they are aimed at improving a small part of a whole, and the learning path can be intriguing at best.

I enjoy getting to grips with new products, but one can only go so far without a goal. My biggest problem with PowerShell was always that it was a reinvention of what most administrators were doing just fine with other technologies. Granted it’s streamlined and feature-rich, but as a fairly hefty departure from Windows command-line scripts or even Windows Scripting Host, the clear need to adopt it wasn’t ever really there while the learning curve was rather steep. This is a big problem; I knew I needed to learn it, but without a problem to solve the effort required doesn’t seem to match the reward.

I’m not the sort of person who will gladly sit through pages of manuals or RFCs to understand a new product or protocol. I’m much more hands-on; my personal systems include a myriad of products that I never imagined I’d come to rely on, they were mostly simply trying to understand how they work. Now that I do depend on them, I have bumped into each of the nasty bugs and side-effects they present, as well as discovering both features that are not in the headline literature and uses that even I hadn’t anticipated when I set out. If you build it, they will come.

And this is the focus of my argument. Pure learning, whether theoretical or practical, has no use in and of itself. Only when technologies are applied do they have value. Storage Area Networks (at least before iSCSI), systems monitoring, ERP applications and even the larger database configurations are beyond the need of the average technical user, and require hardware that should exist outside of a raised-floor, fluorescent data centre. Yet almost every technical enthusiast and support professional I know has some form of lab at home to explore these technologies and the products that offer them.

In my own explorations of technology, I have become decidedly indifferent to the specific products I am evaluating, since they come and go. I am vastly more interested in what’s going on under the hood, since these implementations are much more stable across product versions than the latest trend in user interfaces that mark the biggest visible change in product releases. Using open-source software I’ve been able to emulate and get to grips with almost all of the concepts used in large infrastructure installations, from SANs to firewalling to virtualisation to build and deployment to robust databases, and all in a single server. If my employer found it expedient to spring for a lab to learn about the proprietary equivalents, it would cost in the region of thousands to tens of thousands of dollars, and still I’d only learn more about how to perform specific tasks rather than understand the deeper concepts. Of course, YMMV.

So how do you keep up-to-date with current products, that come and go, while still learning skills you can use professionally? Well obviously the specifics of any implementation is important, and getting to know the interfaces, procedures and maintenance of the products is critical if you’re in the support role. But branching out into other vendors can bring a much deeper understanding of the underlying principles and methods than just installing the latest shiny package. If you are fortunate enough to have an employer that has a good lab, schedule a few hours a week for tinkering, and write the results of your evaluation out.

The modern workplace (especially in IT) is less about protecting your job by hiding what you know than in previous decades, and if you can demonstrate your ability to command a new approach, sharing your experiences can only do you good. I’ve found open IT departments and companies that constantly, and critically, evaluate themselves and the ecosystem they work in are definitely more productive and rewarding places to work.

The Bolt-on Operating System

For years I’ve wondered why on earth so much is crammed into the Windows base image. Sure, they’ve got a decade worth of hardware to support, and since vendors don’t create standardised, reusable code like Linux, this is a significant factor in bloat.

But one of the longest running gripes I have with Microsoft’s OS offerings is that all manners of features are included that I don’t care about – and some that I am super-passionate about are just plain gone! The base installation for almost any major distribution of Linux will include a lot of productivity tools, but leave out some others. It comes down to personal choice, but since all the software is free and available to install from the Internet, this is no inconvenience at all - assuming good Internet connectivity.

That last point is actually quite big, since the distribution may contain an impressive array of software, in the end I’m probably going to want something that didn’t ship on the CD. Can’t have it both ways I guess.

But obviously, the last thing Microsoft would want is to not install something, then have you go back to the installation disc for the features you’re enabling. Of course, they STILL haven’t figured out that Unknown Device from Unknown Manufacturer isn’t a helpful message.

One of my recurring gripes (that is before I started blogging, so I can’t really prove it) is that my Windows Server has a GUI. Seriously, I don’t want a GUI. I want my apps installed on my server, and the management interface installed on a workstation somewhere else. Or the way Linux handles it, X as a process, in my own privilege space that can be launched just for me, and perhaps VNC or X-like somewhere else. The problem of course is that almost every Windows application requires a GUI to install. Sure, MSIs offer silent installs, but so often these line-of-business apps don’t have neat MSI or respect the silent option.

There are two things I’m getting at here;

First, even though the product is named “Windows”, and it’s grown out of a desktop graphical OS that pretty much reinvented the way we deal with computers (though much cred to Apple), I want a server. No people standing in front of it, so no pretty colours required. All the prettiness should be produced by the apps and displayed as IP packets. And on the same note, I see a bunch of sound drivers hanging around.

Well, Windows Server 2008 has the Core option you tell me. It’s a good step in the right direction, but I was genuinely disappointed to see the command prompt surrounded by a window. MMC is still there. I had actually expected to see a text console only, no graphics. Quite simply, it’s a waste of resources. Any app, service or component worth its’ salt is manageable remotely, from simple DHCP up to complex SQL Server clusters. It’s also a danger, as I recall at least one major outage at a previous company thanks to faulty graphics card drivers from <server vendor name censored>. And the GDI component, dating back to Windows 95, handles page and print rendering. Exploits in the 16/32 bit era and in the 32-bit era come to mind. That last one spans products over three release generations, all for a component that doesn’t belong there in the first place.

If I want graphics processing, print rendering etc, then let me add it on later, the way I would Ghostscript on Linux to make a PDF. That brings me on to my second point, and the trust of my argument: components that don’t belong.

The Sasser worm devastated computer estates around the world, by exploiting code hooks in the LSASS.exe process that handles security arbitration between requesting apps/users and security providers. Read closely into the articles, and you’ll notice that it is specifically a problem with code for dcpromo, the process that turns Windows Server into a Domain Controller. The applicable hotfixes patch the code on Windows Server.

On XP, the hotfix removes the code.

Just what was it doing there in the first place? I know the server and desktop products share a codebase, but this irks me. I’ve personally implemented (though not used) a hack to enable RAID on Windows XP that officially doesn’t support it, since the raid driver and all the GUI code is present but disabled. I suspect the same is true of the EFS code in all Home versions of Vista (since you can read EFS-encrypted files from Windows XP upgrades just fine).

Windows Server 2008 requires you to specifically add features to your server before activating them, like AD Directory Service (AD DS), and installations of SQL Server and Exchange (at least) check for updates to the installer before running, getting them closer to the model of Linux distributions – adding a feature from the online repository ALWAYS adds the latest version.

In the end, the development models of these two are very different, so I’m keen to see what further advances can be made on both sides. As always, security and functionality butt heads, somehow I end up with the headache…

Virtualisation of basic services

So, virtualisation is everywhere. It departments are running more and more of their workloads on virtual hardware, and in the data centre that’s a big thing. It’s an accepted fact that most servers in an organisation run at near idle all day long, and even core systems like directory and file servers barely crack a sweat serving their users. For small and medium-sized entities (yes, they have datacentres, just not raised floor, air-conditioned ones like the big guys) this is a problem.

I was asked a while ago why I enabled file compression on my laptop, and seriously considered it on file servers. A few years later another query was why I enabled full-disk encryption on the same piece of kit. Doesn’t it affect the responsiveness?

The simple answer is yes. And no.

CPUs, memory and disks today are so, so fast that there’s not much they can’t do for an individual user. when asked what laptop a person should buy, I recommend the pink one if it’s a girl asking, and jet-black for the guys. Seriously, laptops today can wipe the floor of a desktop from five years ago, and last I checked we’re still doing the same things as back then: browsing the web, writing emails, laughing at cats. If you’re a developer, gamer, or serious showoff, you probably have a dedicated desktop for the hardcore tasks, and the point remains: Moore’s law has seen to that.

Of course, anything you put in the path of data from the disk to CPU/memory is going to slow things down, but this comes back to the more important point: Do I notice?

When I’m running a compile job, chances are the disk’s head movements trying to find each source file and library is going to be the first bottleneck. Then, if it’s a biggie like mplayer, the CPU is going to be loaded for a while doing the compile. Encryption? Oh, I didn’t even notice it’s on.

This brings me to my point, with uber-powerful, multi-core servers cheap as chips (e.g. HP ML110 G5, Dual-core 2.8Ghz Core 2, 1G RAM, 250G SATA for 350 ex tax, HP.nl website 2009-12-07), do I really need to treat these things like fragile porcelain ware?

So, given a department of 100 users in a remote office, where I need to provide local AD, file services, updates, VPN to head office, backups, what’s to stop me buying a cheap server, loading it up with RAM and disks and virtualising all functions in one (or a few), neat little grey cubes with a reputable brand name on the front?

Now here, I’m thinking of Linux as the hypervisor, but that’s my flavour of choice. Something as simple as AD is a good case, since it’s by definition a replicated, resilient database so failures are not a big impact (authenticate somewhere else), and doesn’t do much in the day. It’s also advised to segregate roles on Windows servers, and for resilience perhaps have a second box if you’re paranoid . That’s one, maybe two little cubes that I can’t share for other functions. And it’s going to consume a few gigabytes of disk space, and even less RAM. A whole cube, just for you? Share!

Linux virtualisation has come a long way, and I consider it to be solid, fast and these days wonderfully easy to administer remotely. KVM, as I use, is just another OS process, no funky drivers or installations and since it runs on a full (and standard) instance of Linux, all the management tools for networking, storage, logging and troubleshooting are available.

One simple example: since all traffic traverses the host’s virtual network switch, all traffic can be inspected by a packet sniffer as powerful as Wireshark. No drivers in Windows, no strange binaries. If everything’s virtualised on one box, ALL the traffic is visible for troubleshooting in one spot. Neat!

Another, consider three servers, each requiring 100G storage. That’s probably six disks (minimum) if you’re doing things right (RAID-1). One one hypervisor host, that’s four 100GB disks (RAID-5), only two if you go the 500G route (RAID-1, plus a spare I suppose). Now I’m also a fan of software RAID. My case above stands, there are other factors more likely to impact performance, no least the client’s ability to pull data fast enough from a file server.

So, Now I’ve one box with my local services running in VMs. Full-disk encryption? Well, Windows 2008 has BitLocker, with all the TPM and USB key requirements. How about logging in to the host remotely via SSH, one of the most bulletproof protocols for remote management, and enable the encryption remotely. With a script. Not one byte in plaintext, not even the partition table. Break that, evil hardware theif!

Snapshotting? check
SNMP/performance monitoring and alerting? check
Hardware support? check
Virtualising legacy systems? check (especially MS-DOS on pesky 64-bit Windows)

Worth a think

The case for open-source virtualisation

I’ve been quite a keen follower of virtualisation since the early days of availability on commodity hardware. Obviously, big rigs like IBM, HP etc have been doing hardware partitioning since time immemorial, but I’m interested in the stuff that lets me run an OS concurrently with my main OS on a desktop.

To my mind, one of the big advantages of Windows (and to a lesser extent Linux) is the homogeneity of the OS on desktops and server. That is, if I write an application, website, database etc on my Windows PC, compile and run it, it should deploy to my Windows Server without any changes. This is in stark contrast to the development model for older big iron systems, where the development happened on the system itself, probably on a dedicated development partition, but quite certainly not on my Windows (OS/2, DOS etc) workstation without cross-compilation.

Nowadays, it’s taken for granted, but that’s always been in the back of my mind. With virtualisation, I can now deploy a test partition on a development server for Windows. It’s not just for development, but for testing too, and in my current path to get my Microsoft certifications up-to-date, it’s a godsend. I’ve got a fairly sizeable server that does almost all of my home tasks; file shares, email relay and filtering, VNC, photo sharing website, music streaming, proxying, VPN, and I’m working on VoIP. The thing is, it’s Linux.

I run Fedora. At one point I banished Microsoft from my home, just to see if it was possible. I wanted to accomplish as much of my home automation and services using open-source software as possible, and put Fedora on my main server (it’s actually been there for ages), Fedora on my power laptop and Ubuntu on my lighter laptop. and hey-ho, it works!

Well, actually, my mileage varied. Ubuntu is just great, it works well with laptop hardware (especially the Intel graphics), wireless and sound. Fedora also just works, that is until I tried to virtualise on the laptop.

I had read about Xen some years back, and how it offered paravirtualisation on Linux. I tried to get that working, but once I figured out it interrupts communications with my Radeon chip (they claimed it wouldn’t) I dropped it. Enter KVM, stage right!

I’ve been a keen follower of AMD’s hardware virtualisation (Pacifica) since the inception. I’ve actually been a very big fan of AMD’s for some time, but that’s for another post. VMware and other players have been doing hardware virtualisation for some time, but the Intel architecture just doesn’t play well with others, meaning the host OS and hypervisor needed to do a lot more work than ideally required to keep up the illusion for the guest. VMware’s intellectual property in the regard is substantial, and for years were the cut-and-dried leader in the field.

Qemu is a mature hardware emulator I’ve been using for years that runs all kinds of CPUs on multiple host OSs, but still runs in user space (i.e. no kernel privileges, and a lot of context switching for privileged ops). As soon as the kernel module is loaded (kqemu), things perform rather well indeed, but still noticeably slower than bare metal.

Hardware virtualisation (HVM) in Intel chips changed that, since now a lot of the grunt work like intercepting privileged interrupts can be caught by the CPU and sent off to the Hypervisor efficiently. In no time at all, the existing Qemu binaries were extended to include KVM, Kernel-based Virtual Machines. I’ve been a convert ever since.

By using a well-established virtualisation platform like Qemu, with excellent hardware support, KVM runs a lot of guests!

So why am I raving about this? Well, I often get asked why a Microsoft techie runs Linux. As I have previously stated, I tinker, and Linux offers me that chance. I get to play with raid in granular detail. Layered on that is LVM, which virtualises storage. You don’t have to get that complex, but it’s the closest I can come to simulating a SAN in my own home, with resilient, abstracted hardware hidden from the VM.

I make and break bridges and virtual networks on the fly. I’ve got three gigabit Ethernet ports, and segregate them by function – to the point where ALL VM traffic is on a dedicated port, so that if a VM talks to the host it pops out through the physical switch and back in on the front interface.

Here’s the key thing: While all of this is probable available from other vendors, and some parts may even be free, I’m in control. Sure documentation can be sketchy, and it requires a lot more basic knowledge of networking, storage and hardware architectures than other solutions, I’m hardly take my car to a garage to be serviced if I wanted to be a mechanic, I’d get dirty. Break things definitely, but that’s a great learning tool.

My power laptop used to break whenever I entered standby, and I mean trash the root file system (ext3 to boot, supposedly bullet-proof) so I filed a bug. As it turns out, it only happened when I was running HVM, so I applied a patch published 30 days before to my kernel. Voila!

Now this is a big thing! Without being a paid customer of Redhat’s, I got a problem resolved quickly and comfortably. I must stress here that Fedora is BETA software, permanently, and I expect it to break, but frankly for my needs it’s just fine, and when I can get things like this resolved, even better.

Sure, it’s painful, but it’s all under my control. I’ve scripted Windows 2003 deployments with snapshots that boot a new instance, sysprepped and ready to go, in 7 minutes. IPTables lets me simulate complex networks, firewalls and even lossy links.

I remain firmly impressed!

So why am I so keen on Linux?

I've been trying to dissect my career, certifications and interests, and how the three don't really match up.

You see, I'm a Microsoft certified Professional (and now Technology Specialist, soon to be IT Professional, exams, exams, exams). I have been since 1997, when I first sat the Windows 95 MCP exam. I didn't really know what I was doing it for, except that employers wanted to see that sort of thing. I'd been using '95 since a few months after the release, when I went out and bought a copy.

Since then, my career has moved from technical support on desktops, to the point now where I'm quite comfortable giving advice to large organisations on how best to spend a million or so dollars (Euros, etc). This is not small-time stuff, I've advised corporations and governments on how to do mergers and streamlining, and how technology can support their immediate and long-term goals. I've solved interesting problems fraught with politics (but I'm nowhere near arrogant enough to deal with that effortlessly), and sometimes I get to have some fun.

It struck me on an assignment in June 2008 that I had zero allegiance. I'd been sent off to see how best a pharmaceutical company could make use of it's new acquisition, a prominent retail brand in it's sector. The big guy had Microsoft Exchange, the little guy had Notes. they kinda knew where they wanted to be after the merger, that it didn't make sense to keep two teams doing the same role on different platforms. Makes sense.

As I dug deeper, I noticed that the e-mail and collaboration part of the story was not all that important. As is the norm for Notes, a little ecosystem of apps had been developed. However, whereas most organisations tend to have a cluster of disparate Notes apps, the small guy had a plan, a comprehensive one too that made an application platform out of Notes, and a rich set of interconnected apps delivering business intelligence, dashboarding, some data mining. Wow.

So in the end, all my credentials (and by then I'd achieved five Microsoft qualifications) meant nothing, since it made no sense to throw notes out, or even relegate it to an application platform and move all e-mail to Exchange. So my recommendation was to ditch Exchange, at the very least for user e-mail.

That's kind of my point. I'm not a standard bearer, but I am a flag waver. My days of technology for tech's sake are behind me. Don't get me wrong, tech is cool and I'm always drooling over the latest gadget or software release, but when it comes to my work it has to mean something, be pointed in a particular direction, support a bigger plan.

So frankly, I'd be negligent if I just supported Microsoft's plans. Or any other platform for that matter. It disappoints me to see MS and FOSS guys going head-to-head. I enjoy using Linux for the same reason people tinker with classic cars. Sure, the new model from the dealer is likely to be more fuel-efficient, safer, better riding. But I like to get my hands dirty, and you can't do that with most modern cars.

I don't know if I've made a point here, but maybe I don't need to.