Articles with tag - Virtualisation

18.03 20110

Black Hat EU: You are Doing it Wrong: Failures in Virtualization Systems

Wrong Way ... Way Wrong a CC NC SA image from Bob.Fornal's Flickr stream

Wrong Way … Way Wrong a CC NC SA image from Bob.Fornal’s Flickr stream

By Claudio Criscione

Virtualization aims to save money, make things simple and quick to deploy. Saving money and quick deployment are arch enemies of security

Virtualization products require security on the hypervisor level. Being able to hop from one virtual machine to another is not acceptable. Also there are a lot of products that focus on the security in the virtual machines, but virtualized infrastructure are complex by nature.

Relative lame bugs like XSS can be a big deal in virtualization infrastructures

Claudio demonstrates that live on stage, by exploiting a XSS bug in VMWare vCenter which took 1.5 years to patch.

Claudio showed us how an unprivileged user on the vCenter machine able to read a logfile contain the administrator SOAP session ID. Using this ID and Vasto administrator privileges where obtained. Until the last patch read-only access to vCenter meant that the user could take over the virtual infrastructure using standard tools.

Next attack demonstrated is against an Oracle virtual machine. Using standard “lame” exploits Claudio was able to hope from the application level administrator to the system root account.

So there are still some very simple vulnerabilities in this software.

Virtualization software is broken today, and we have to treat it accordingly. We have to make people aware that it is broken.

Virtualization infrastructures should be setup in such a way that a XSS in the management layer cannot lead to a disaster.


read more
28.10 20100

How I experienced VMworld 2010

Like over 6000 other people, myself and two other colleagues went to VMworld 2010 two weeks ago, in Copenhagen. This event, one of the largest vendor organized events in the world, is three days packed full of all things VMware (and virtualization) related. There are a lot of exellent posts out there already offering specifics on the subjects presented. This post is not meant to replicate that, but offer some insight into my experience there. Three very important aspects outside of the normal presentations for me were the Solutions Exchange, the VMware labs and of course networking (social, not the hardware kind).

Solutions Exchange

First of all, let me start with the Solutions Exchange. With over 113 vendors showing off their wares, iPad give-aways everywhere, booth-babes distracting you and booth staff trying to get your attention this can be very overwhelming at first. However, you are there for three days. Because of this, you have the time to walk around, get to know who is where and pick your interests. What also helps a lot is the fact you have 30 minutes between sessions. Going from one session to another takes maximum 5 minutes, meaning you have 25 minutes to spare. While that is a limited time, the vendors know this as well and keep very short presentations on a plethora of subjects related to their product. Added benefit is you don’t have to sit through an hour of slides and sales talk, they get right to the point.
In the end, I got a lot of value out of the Solutions Exchange. Of course I had some companies I knew I wanted to visit beforehand (Veeam being a good example). A lot of others however I might not have visited per se (like the VCE alliance with their awesome vBlocks) or hadn’t even heard of (like Nimsoft with their monitoring solution).

VMware labs

Another aspect I was very anxious to try were the VMware labs. The labs offer you the unique opportunity of trying out any and all VMware products in a pre-deployed environment. All labs are virtualized in a big cloud. Even the ESX(i) servers were virtualized and then deployed on demand. This is ‘eating your own dogfood’ in the purest form. The disadvantage here was the performance: my first lab was almost unworkable the first ten minutes. After that time, it was still slow but workable. With a couple of thousand VM’s running off of a cloud in the USA this can be expected though I guess.
The physical lab setup is quite nice: it is a dual-screen setup, where the actual labs are on your left screen. The right screen will have the lab instructions, any remarks and tips and this will allow you to not keep having to switch back and forth between screens. For those with a dual-screen setup, you know what I mean (personally, I’ve arrived at three screens which is even better ;) ).
Some labs are free-form. I got to try CapacityIQ, Chargeback and Orchestrator in the Sandbox for example. The sandbox was a complete vSphere environment, with all the bells and whistles installed and configured. Since I can’t spend the time on doing a Proof of Concept per se, being able to play around and mess with the config proved to be invaluable for me. The lab instructions here consisted of some pointers but the idea really was to mess around and try out anything you wanted. Again, this was not some kind of demo but an actual environment created just for you.
Other labs are more strict (using strict very loose, you can still mess things up if you want). I did the installation lab for VMware View 4.5, to see what is was all about. The lab instructions will guide you through the install step by step, clearly describing the actions needed to complete. Even though I was unfamiliar with this specific product and how to set this up, I felt very comfortable going through this and ended up with a working View setup in under an hour!

General networking

This is the biggest VMware related event in Europe. The who’s who in virtualization land is here. Most of the top Dutch bloggers (like Duncan Epping, Eric Sloof, Gabrie van Zanten, Arne Fokkema, Arnim van Lieshout, Joep Piscaer, etc etc etc), a lot of the 3rd level guys from vendors you normally don’t get to talk to a lot and of course big names in VMware like Paul Maritz and Steve Herrod. If there is any chance to shake hands and meet these guys (and girls) in person, it’s here! This is one of the fields where I would take a different approach next time I visit VMworld though. While I had my sessions and labs planned out, my networking was a lot of free format. This works for meeting the vendor people (they aren’t going anywhere), especially the bloggers are a lot harder to shake hands with. It’s not that they have a security cordon around them, but these guys are popular! I hate to intrude on an ongoing conversation but in the end this meant I did not get to shake as many hands as i had hoped for. Next year I’ll be making sure I have at least pinged some of these guys beforehand, so they expect me to walk over and shake hands. Makes it less of an intrusion and more of a friendly talk.


In the end I still spent most of my time visiting the sessions. As I mentioned earlier there are already excellent posts online on all of them (most Dutch bloggers were posting real-time, check Gabes Virtual World or NT Pro for example). I will do a quick summary of what I visited and hope you get some value out of that.

  • Keynote
    The keynote was done by Paul Maritz ad Steve Herrod. These guys know how to do a presentation but there was of course a lot of marketing. What it boils down to: Cloud Computing is not just a marketing term. Their goal is to make everything you could consider a resource (storage, network, processing power, applications, etc) available as flexible as possible. This allows you, the administrator, to offer IT as a service to the end-user. It makes his experience transparent and gives him the flexibility he requires. Sorry, can’t explain it any better with less buzzwords, feel free to correct me in the comments ;) .
  • Troubleshooting using esxtop
    esxtop has always been a powerful tool for troubleshooting your problems on an ESX server (and of course resxtop for ESXi). With the release of 4.1 new counters are shown (note: values were already recorded, simply not shown yet). Some of the most importent ones are LAT_C and LAT_M, showing CPU and memory contention. Don’t forget: you can export esxtop into csv and analyze further with perfmon or esxplot (open source, available at VMware labs).
  • VMware View 4.5 Technical Overview
    VMware View 4.5 offers some major improvements over the previous release. Most important are full support for Windows 7 32- and 64-bit, PCoIP authentication with smart card and offline support. The last one is a major performance improvement for road warriors and now allows them to work without a connection.
  • Intelligent HA: Application awareness with VMware HA
    VMware teamed up with Symantec to bring you this one. It boils down to application clustering support for applications with the intelligent layer of VM underneath. It is based on the Symantec clustering product and the years of experience they have there.
  • Planning and designing a HA infrastructure
    This session, given by the awesome Duncan Epping, focused on all those little (and big) things you need to keep in mind when using VMware HA. If you regularly read his blog you will know most already. Some highlights: keep in mind you can only have 5 primary nodes in a cluster, design for management network redundancy (a VM network is not a HA network!), change your failure detection values when adding a secondary Service Console. Very important improvements in 4.1 are the fact you are now able to see the master status of nodes and in the vCenter client you have a Cluster Operational Status. Furthermore, in 4.1 U2 detection for lost lock and thus partial split-brain has been built in. Duncan also discussed an often missed ‘Golden Nugget’: vm monitoring. This allows you to quickly recover even from a BSOD and makes a screenshot for you to ease troubleshooting. On the roadmap: storage hearbeating, no more DNS dependency and Improved isolation response actions.
  • Transitioning to ESXi
    We’ve all known this was coming and 4.1 is the last release with a full-blown Service Console ESX. ESX is dead, long live ESXi! Whereas with 3.5 and, to a lesser extend, 4.0 not everything could be done with ESXi, those days are gone. With the focus on agent-less management via API, CIM and PowerCLI a powerful framework now exists for ESXi management, more powerful than ESX with the agents in the COS. Worth noting: Tech Support Mode (TSH) local and via SSH is now fully supported, with all commands sent to syslog. You can still disable it and go a couple of steps further with lockdown mode.
  • 10 best free tools for vSphere management
    Every VMware administrator worth his salt should have these installed! For a great list check Kendrick Coleman’s blog, but the ones mentioned were: VMware Guest Console, Veeam FastSCP, Trilead VM explorer, XtraVirt vSphere client RDP Plugin, vEcoshell with the Community Powerpack, VKernel CapacityVIEW, vSphere mini monitor, RVtools, vFoglight QuickView and Xangati for ESX.
  • Tech Preview Storage DRS
    Honestly, this was one of the coolest sessions I attended mostly because of the subject. While CPU and Memory have been a resource where your VMware cluster is able to spread the load, storage is coming too. This is a complex challenge (as shown in the presentation) but in the end this will add more awesomeness to your VMware cloud.
  • PowerCLI for administrators
    This session, given by Alan Renouf and Luc Dekens (those guys are awesome) was actually not new to me. I’ve heard Alan spread the word on the awesomeness of PowerCLI before so I didn’t need convincing. What it boils down to: use PowerCLI to make your life easier as an admin. Like the tagline on Alan’s blog says: “Everything is poshable!”
  • SRM using NetApp
    SRM is a workflow task allowing you to script your Disaster Recovery and actually test it. I am impressed by the product and would love to deploy it as I see some very good use cases. There is a but: it’s hideously expensive in my opinion and I simply can’t get the business case for it worked out. Too bad.
  • Best practices to increase availability
    I’m afraid to even summarize this session. It’s given by two people who I consider absolute superstars, being Chad Sakacc (EMC) and Vaughn Stewart (NetApp). The fact alone these guys, who work at competing companies, are able to give a presentation that will grab your attention for the full hour is amazing! The presentation was packed full of tips and tricks. One I want to pick out: CHECK YOUR MISALIGNMENT (yep, that is supposed to be in capitals). They took the time to explain and emphasize this can cost you a lot of performance. Something else they mentioned: there is no ideal protocol. Each protocol is different and has it’s own super-power and kryptonite. What works best is designing for those.
  • VDR, all you need to know
    VMware Data Recovery is the backup product you get for free when you have an Enterprise Plus license. While this started as a very simple solution, it has grown out to be very powerful and a ‘good-enough’ solution for a lot of people. As long as you are aware of it’s limitations (and use the most recent version) it could very well be enough for you as well.

Concluding, VMworld 2010 Copenhagen has been an awesome experience for me. I’ve gotten an incredible amount of information, packed in three days. Some of that is not directly applicable to how I am using VMware now, but a lot of it I can apply directly to my day-to-day work. As such, I’m hoping I can make it again at the next VMworld!

read more
15.04 20103

BlackhatEU : Virtual Forensics

By Christiaan Beek

From isfullofcrap Flickr photo stream. Creative Commons License

From isfullofcrap Flickr photo stream. Creative Commons License

BlackhatEU : Virtual Forensics
By Christiaan Beek

What are the challenges when you have to do forensics on a virtual environment?
•    What are the tools available?
•    Are the tools forensically sound?
•    Where is the data?
•    Who owns the data?
•    What forensic techniques do we use?
•    How to acquire data from the cloud?

Citrix is a nightmare for forensics investigators. There is no personal hard disk to investigate, only a personal profile which does not have very much data in it.

read more
29.09 20090

What startrek tells us about the future of IT security…

Robert “RSnake” Hansen has written a wonderfull peace about what Startrek can tell us about future IT security

Virtualization security is an oxymoron – even in the distant future: I mean, really, how many times has the whole damned ship been taken over by some overzealous holodeck character? Whoever wrote the holodeck hypervisor really needs to be put in a room with Warf for a few hours so he can explain with his batleth what the need for true physical and logical isolation is. Why some Sherlock Holmes character should have access to main memory, I’ll never know. Too bad we aren’t smart enough in the distant future to think about hardware isolation instead of relying exclusively on dangerously faulty software.

You should really check it out have a laugh and then think about it…

read more