VSA Deep Dive - The Lab

VMware VSA DeepDive – Part 1 – Building the lab

My deep dive into how the VMware VSA works was really brought on by necessity.

On the surface, it sounded like a great solution – and to be fair it does work quite well overall. It acts as a RAID-1 of local storage over Ethernet which is a life saver for cost-conscious companies or the SMB space. And it is really designed as a fire and forget solution – if you don’t plan on patching your hosts or really making any changes to things ever.

Up front, your investment seems like a good choice – in some cases it may have been the only one. (I’ll let you figure out where I sit on this.)
But as a virtualization admin or engineer, you will have to update it, patch it, and do maintenance at some point. For one location, it’s easy.

For hundreds of locations, it gets a lot more interesting.

The Lab Requirements

In short, you need a minimum of 2 ESXi hosts running 5.1 or higher, a Windows vCenter 5.1+ installation, and some local disks. For my home lab setup, I’m just building nested ESXi with a single thin provisioned 100gb disk per host – that’s enough to get you going for building this out.

Keep in mind, I’m setting these nested instances up in VMware Workstation!

You will need 4 vmnics on each host – 2 bridged or on the same network as vCenter, and 2 host-only for the VSA replication portgroups. The vmnics must be in this order by default:

  • vmnic0 – Bridged
  • vmnic1 – Host-Only
  • vmnic2 – Bridged
  • vmnic3 – Host-Only

If you are asking “why?” at this point, don’t worry – it will all make sense. I’m a professional.

Once you have vCenter running in Windows, you will need to install the VSA Manager plugin. It is Windows-only, and must be installed on vCenter itself, hence the earlier requirement.
The only gotcha is to make sure that the account you are logged in as on vCenter has admin privileges to both the box, and vCenter inventory.
Other than that, it’s pretty much a next-next-next type of deal.

Add your 2 ESXi hosts to a new datacenter object, and you are almost ready to get started.

Team Make It Fit

By default, the VSA will do a check on the host hardware to see if your RAID controller is on the HCL. For obvious reasons using Nested ESXi isn’t going to work exactly as planned.

The solution is simple – just disable the auditing of the host during installation entirely!
On the vCenter Server, go into the VSA Manager config folder and find the ‘dev.properties’ file, typically it’s found here:

  • C:\Program Files\VMware\Infrastructure\tomcat\webapps\VSAManager\WEB-INF\classes\dev.properties

In this text file you’ll see a number of tweaks you can perform to suit your lab. The value we care about though, is this:

# Other operations

Simply change the value for host.audit to false, save the file and restart the vCenter Management Webservices service on the server. Then restart your vSphere Client. When you run the wizard next time, you should only get a warning, and not a fatal error about how totally unsupported you are. It’s a shame we can’t say the same back to it, at least until September, 2018.

Click the datacenter object and you should see a tab called VSA Manager. All operations are done from this menu – it doesn’t exist in any other context. You’ll get a certificate warning, even if you have a signed internal certificate installed for vCenter – so just accept it. This is only the first crack in the armor!

The installation should be pretty straightforward. You’ll need to specify a cluster IP, and a cluster SERVICE IP since you’re running 2 nodes.
When you installed VSA Manager, the clustering service was installed on vCenter, so just point the SERVICE IP to your vCenter server.

  • On a remote / ROBO deployment, you can run the clustering service on another VM – there is an available clustering service installer in the VMware portal for Linux or Windows.

The cluster IP itself can be whatever you want as long as its on the same network as vCenter, or the Cluster Service in the remote location.

In about 15 minutes, you should hopefully have a functional VSA installation ready to rumble for your lab to tinker with and destroy!

A completed VSA Lab setup.
A completed VSA Lab setup.

And destroy it, we shall.

Introducing: My SDDC 2015

First things first. When you’re in this line of work, and especially for engineer or higher levels, you have to have a lab. It’s pretty easy to get one going, especially if you have a garage where you can put your baremetal boxes, with your own Catalyst switches, and all that fun stuff.

Living in a 500 square foot Seattle condo poses some interesting challenges when it comes to a lab. I have a spouse, 3 cats, and like many people, a gaming habit. Given my space constraints, I had to go small and practical.

So, here’s what I roll–what I call SDDC 2015. I used this for preparing for my VCAP5-DCA exam and it worked just fine.

The Case

The case.
The case.

I don’t think I could be much happier with a case than this one, the Lian-Li PC-V359WX. For $170 at the time of purchase, it seemed steep, but I wouldn’t change anything. It’s hard to find now, but I can’t find really anything to fault on it. It’s a two-story box, with Plexiglass on top letting you peek inside to see your motherboard and all the goods. You can even go SLI/CrossFire if you have adequate power, but I haven’t needed that.  Yet.

This case fit the dimensions of my current TV stand perfectly (although, a bit tight!) and the reviews were glowing. I have been a fan of Lian Li cases for a while now, but never bothered to really look at them because of cost. This time, it was spare no expense!

The Build

Intel Xeon E3-1230v3
Intel Xeon E3-1230v3

The brains of the outfit – an Intel Xeon E1230v3.

I wanted a real CPU to handle this lab – again, I had to pack as much punch as I could in one chassis without blowing my budget. This CPU is fantastic from a price/performance perspective. 8 Cores, VT-d, all the bells and whistles, and it runs quiet and cool.



The ASRock Z97M.
The ASRock Z97M.

The motherboard as also important in terms of feature set. I didn’t want a piece of junk Realtek NIC running my VMs, much less on my network. But, I’ve got a size constraint and that narrowed the list down to just a few candidates, and I decided to go with the ASRock Z97M. This one is great – it has HCL-supported Intel NICs, so for nested virtualization purposes, less drama with building custom ISOs. Add in 32GB of DDR3-1600 memory from Crucial, and that’s plenty of room to run things!


The Drives

SSD – Crucial MX100 512GB, another coming soon!
HDD – 2x Hitachi 2TB 64MB Cache – RAID1

The drives are really the biggest item on the list in terms of quality of life. SSDs make a lab so much more bearable to work with it’s crazy – don’t skimp here is my recommendation. You just thin provision on the SSD for speed and you’re set. The Hitachi HDDs are just there because I had them from my old build, and the other 5 went into my home brew NAS, which performs quite a bit of extra duty.

The Software

When I call this “SDDC 2015” it’s really more of a joke – this lab runs Windows 7 Professional running Workstation 9, which I got for free after passing my VCP5. Some edits to Virtual Networks later, and I now have nested ESXi 5.1, 5.5, and soon 6.0 hosts. Some additional nested VSAN instances, VCO, along with the usual AD and DNS servers.

And when I’m done, my wife knows how to startup Netflix and Plex. This is more important than you realize!


Such compact, wow
Such compact, wow

It’s really not a lab until you have something that can serve up blocks!

Once again, I went with Lian Li’s cases in Mini-ITX. This guy is the Lian-Li PC-Q25B and it is amazing. The motherboard is an ASUS H87, with an Intel G3220 CPU. The setup is really overkill for what it does, but I have put it to work as a media server, and my wife makes sure it’s always working!

The OS is FreeNAS 9.1.2 to a USB stick, and I have 5 Hitachi 2TB 64MB disks in there as a RAID-Z2 (essentially RAID-6). So far, it works pretty well and reliably.


I’m not sure I will refresh this lab any time soon, so it’s got to last. Maybe if I get a garage I can get myself a used ASA and a stack of 3750X switches.

But, until then – I’ll fire up Crypt of the NecroDancer. I love this lab!


Learning vCO – Beginner Actions, World 1-2

Previously, I went over a very simple example of creating a VCO Action from top to bottom. Combining two input strings into one is quite the thrill ride, but let’s see if we can’t do some expanding and exploration of what we can do with Actions.

Suffice to say, Actions can be as simple or as complicated as you require. In this post, we’ll build a more complicated Action for a more specific use case. There’s a little JavaScript in here, but don’t worry – it will be well commented and should be easy to understand.

The Case

A pretty common issue as environments get larger is managing to keep VMware Tools up to speed. A simple Action you could create would dig into a VM and tell you whether it is current or not – and you could integrate that into a larger workflow that could mount up the VMware Tools and initiate an upgrade during a maintenance window.

The Action

From the Actions tab in VCO, go to your module and right-click, choosing to Add Actions.
Let’s name it something obvious…


There are a number of ways to approach the creation of this Action, but let’s say that our input parameter inputVM will be the VM itself (that is, type VC:VirtualMachine) and the return type will be a boolean, indicating whether the VMware Tools are up to date or not.


With the ins and outs defined, it’s time to write out the code. If you’re inexperienced with JavaScript or the vSphere API in general, prepare to meet your new best friend, the API Explorer. It is accessible within any scripting window, or from the main client menu under the Tools option.

Accessing API Explorer from the main client interface.
Accessing API Explorer from the main client interface.

The Explorer contains a comprehensive look at every object type – its properties and methods, a hyperlink to the types that they both expect for execution, and what they will return when called or accessed. You can drill down deeply into each object type and see what you can extract from it, and also find out what you need to build custom objects to pass to vSphere / vCenter’s API.

To Search the API Explorer, click the magnifying glass above the API Explorer.

API Explorer inside of an Action or Workflow.
API Explorer inside of an Action or Workflow.

Since you want to find out what you can get from a VC:VirtualMachine object type, search for that. When you double-click it, the parent window finds the object you clicked. It’s a little clunky, but you’ll get used to it. You can click the little arrow to the side to expand the VcVirtualMachine type and see what it has to offer.

API Explorer - Searching for type VirtualMachine.
API Explorer – Searching for type VirtualMachine.

As you can see, lots of potential goodies for you to automate or key off of.
In this case, we’re just trying to find the VMware Tools status, so go down the VcVirtualMachine property list until you find the guest property.

The Guest Information property of VcVirtualMachine.
The Guest Information property of VcVirtualMachine.

Notice at the bottom, it shows the return type of the property of inputVM.guest is VcGuestInfo. If you click that, it will take you to that type definition, and you’ll be able to see what other properties are available. Inside the VcGuestInfo list, there are a number of properties that will give you data about the status of VMware Tools. In this case we’re looking for the toolsVersionStatus property.

API Explorer - Finding the Tools Version property.
API Explorer – Finding the Tools Version property.

This property is a string, but if you look at the text, it will tell you where to find the possible values – in this case under the type VcVirtualMachineToolsVersionStatus. Click that link to see the available values that this property can return.

API Explorer - The Virtual Machine Guest Status Codes.
API Explorer – The Virtual Machine Guest Status Codes.

You now can see what the possible values are and what they represent in each situation.
So after drilling down in the API Explorer to get this data, how do we work with it?

The Code

All you really need to know is, you can access the value of the VMware Tools status using this code in your Action:

System.log("The VMware Tools status for the VM is: "+inputVM.guest.toolsVersionStatus)

It’s really that simple, at least for this particular property. With that in mind, let’s make a couple of different versions of the Action.

First Pass

You can use a simple bit of code here – this just says if it’s not current, return false. If it is current, return true.

if(inputVM.guest.toolsVersionStatus == "guestToolsCurrent") {
  // the VMware Tools are current and up to date.
  return true;
} else {
  // the VMware Tools are not where they should be.
  return false;

Pretty simple right?

But what if you want to go a little further with additional checks?

For example, a VM that isn’t powered on will be able to execute the above action, but it will just say “guestToolsNotInstalled” – which may or may not be true! Let’s re-write it a bit using some more exploration of the API.

Second Pass

In the API Explorer, go back to the VcVirtualMachine type that we started with. There’s nothing that obviously tells you the current power state here, but if you dig you’ll find the runtime property that mentions power state, leading you to the VcVirtualMachineRuntimeInfo type.

Keep drilling, and you’ll find the powerState property, giving you a list of all possible states. Let’s use this to make our Action a little more effective.

// is our VM powered on?
if(inputVM.runtime.powerState == "poweredOn") {
  // it's powered up, so let's check the tools info
  if(inputVM.guest.toolsVersionStatus == "guestToolsCurrent") {
    // the tools are current!
    return true
  } else {
    // tools are not current
    return false
} else {
  // the VM isn't powered up, so let's return false
  return false

You can just keep building on this Action if you want, or do what I do: keep it simple.
Actions and Workflows are easier to debug if your code is simpler!

Let’s add this action into a workflow with some basic decision making logic.

The Workflow

Go to the Workflows Tab, open your folder and right-click, choosing New Workflow, and name it.

The new Workflow for your Action.
The new Workflow for your Action.

After you do so, you’ll be in the Workflow Designer – choose the Schema Tab.

In the left menu, search for your Action, and drag/drop it into the Schema.

Adding the Action to your Workflow Schema.
Adding the Action to your Workflow Schema.

After that, you will again get the prompt to setup the parameters within the workflow. Click Setup, and configure it like the below:

Binding the Action's inputs and actionResult to your Workflow.
Binding the Action’s inputs and actionResult to your Workflow.

This automatically connects the inputVM parameter as an Input to the workflow, and also configures an Attribute called toolsStatus that is bound to the output of your Action. Easy!  Keep in mind, you can name the attribute whatever you want!

Once that is done, we will add a Decision element to the workflow. This allows us to create a fork in the road based on the value of an attribute, which in this case is whether the Attribute toolsStatus value is TRUE, or FALSE.

Search for a Decision element in the left window, and drag/drop it after the Action in your Schema.

Your schema, with the Decision element.
Your schema, with the Decision element.

Once the element is added, click the pencil icon above it to edit the settings. There isn’t much to change here, so go to the Decision tab. Click the link there to choose which Attribute in the workflow you wish to check on. It should be toolsStatus in this case – no binding is required for this element.

Setting up the Decision element.
Setting up the Decision element.

You now have a Workflow that can act based on the output of your Action. A few ideas to try:

  • Add the built-in workflow to Upgrade Tools, or Upgrade Tools at Next Reboot on the FALSE workflow path
  • Write a log entry indicating the tools are up to date on the TRUE workflow path

Setting these particular capabilities up are drag and drop affairs, just like adding your Actions and Scriptable Tasks. I realize it sounds lame, but the only limit to what you can do is imagination – you just have to consider your use cases and impact of your design. If you want the workflow to automatically install upgraded tools, can it be rebooted right after if needed? There are many things to consider.

In the next post, we will dive a bit deeper into working with Arrays of objects, such as ESX Hosts or Virtual Machines. Thanks for reading, and hopefully if you’re just getting started with VCO, this was helpful!


VCO – All about that Base64 Action

In my time working with integration with other systems with REST APIs – just about every system using Basic Authentication needs a Base64 encoded string for the password. I figured, why not just make this an Action that I can reference easily later? Spoiler alert: I reference this Action a LOT.

The Code

The Base64 function is culled from various online resources. A few minor modifications, and we have an action that takes input of the string to encode, and a boolean of whether you are encoding the string, or decoding it. That way, we put both capabilities into a single action!

This particular version references the JSFiddle code from here: http://jsfiddle.net/gabrieleromanato/qaght/

The code is unmodified except for the final lines attempting to write into a document – which are removed. A slight bit of logic in VCO later, you have a simple encode/decode Action for your use.

Of note, if you have other JavaScript functions like this you want to use in VCO – this can be a template of sorts! Hopefully you find it as useful as I have.

You can download a ZIP of the Action for import into your VCO right here.

Learning vCO – Beginner Actions, World 1-1

As David Hasslehoff would say, you need some action(s)!

In Orchestrator, Actions are akin to functions in typical programming languages – in this case, Javascript. Functions and Actions can run the spectrum in terms of complexity – some are extremely simple, others can be extremely complicated. But ultimately, if the Action is a repeatable task you’d rather call than write over and over again, making an Action is the sensible thing to do.

Out of the box, VMware piles on numerous Actions you can use in your workflows. These can range from typical things you want to pull from vCenter, such as getting a list of all VMs in a cluster, or getting all clusters in a datacenter. Others can get you a list of VMs that are running a particular OS, such as Windows or Linux.

This post will be the first in a series of creating basic Actions, moving to the more complex, along with integration into a workflow.

The best way to learn the value of Actions, much like anything in Orchestrator, is to work with them or create them regularly.

NOTE: If you are not familiar with JavaScript and wish to use Orchestrator, I would recommend getting familiar fast!
The best online and free JavaScript training I have seen is CodeAcademy. It can take up to 10 hours or so, but you work at your own pace, and get fantastic feedback as you go.

Time to get Modular

The first thing to do is to create your own module in Orchestrator. Creating a module is useful, as it allows you the ability to call your Actions from pretty much anywhere, and export/import them wherever you like. Plus, all the cool kids are sharing their Actions on FlowGrab, so get on it!

From the Design menu in the Orchestrator client, go to the Actions icon, which looks like a gear icon.

The vCO Actions Menu.
The vCO Actions Menu.

In here, you’ll see piles of Actions for many vSphere objects and other types, based on the plugins you have installed. Browse around and look at them – you’ll see many of them are not particularly complicated scripts, but it does vary. Our example in this post will be VERY simple, but it will give you a good sense of how to both build an Action, and call the Action, from top to bottom.

So, let’s create your own module. The typical scheme is a reverse FQDN format. In the screenshot above it is A.NSFV.LAB.REPOSITORY. I could have followed the domain, but I like my stuff at the top!

With that done, right-click the module you created and choose Add Action. Name the Action combineTwoParameters.

Creating your new Action.
Creating your new Action.

Once it is created, you will immediately be taken into the Action for editing. When creating an Action, it’s good to have an idea of what is going in and what you are expecting will come out of it when the code is done running.

Setting up your Action.
Setting up your Action.

In our case, we are doing a simple combination of two values we input into the parameter listing, so the setup and code should be very easy.

Click the 03-edit-action-arrowicon icon above the parameter listing twice. This will create two input parameters named arg0 and arg1, of type String.

Once you have created the parameters, click the Return Type option (which is void by default), and choose the type String. This is probably one of the most important things to do – if you do not specify the return type, the Action will always return nothing!

Adding parameters to your Action.
Adding parameters to your Action.

With that done, below the parameter list will be your code window, as seen here with the API Explorer on the left.

The Action code window.
The Action code window, and your API Explorer.

Now, you’re ready to code the Action itself.
Since we’re keeping it simple for this example, you can copy/paste the code below.

var output = arg0+arg1
System.log("The combined value: "+output)
return output

Admittedly, that’s pretty easy! We’re taking the arg0 and arg1 values and just combining them, assigning the value to the variable output.

Then, we are returning the value of the output variable.

Once this is done, click Save and Close, and go to the Workflows tab.
You’re done! But now it’s time to show you a couple of ways you can use your new Action.

Using your Action in the GUI

The first way to use the Action you created is to use the GUI and add it to a workflow.

Begin by right-clicking and choosing Add Folder. Create your own folder of choice, outside of the standard workflow library.

Next, right-click and choose New Workflow. I named mine based on the post, but obviously, you can name it to your choosing.

Long name for a simple workflow.
Long name for a simple workflow.

When you have created the workflow, it will take you straight into editing mode. Click on the Schema tab.

A workflow's blank slate.
A workflow’s blank slate.

In the search bar, you can type the name of your Action and it should show up. Click and drag it into the workflow area and place it on the arrow.

Adding your Action to the new workflow.
Adding your Action to the new workflow.

At this point, you’ll see a message pop up about adding the “activity’s” input/output to the workflow. In this case, click Setup to do so.

Setting up Action and Workflow bindings.
Setting up Action and Workflow bindings.

This menu can help get the values of your action assigned to the workflow more efficiently, or mapped/bound to other attributes.  For this exercise, arg0 and arg1 can be set to Input, and the actionResult can be set to Local Variable. Then, click Promote.

Now, you can run the workflow, and give it a go.

This should be pretty easy.
This should be pretty easy.
Run that bad boy!
Run that bad boy!

The Trace View should output the value of the two strings added together.

The moment of truth!
The moment of truth!

In addition, the value has been assigned to the workflow attribute actionResult, and you could bind that attribute to other elements in the workflow as an argument or parameter.

Using your Action, in a script

I indicated earlier that Actions can be called in different ways.  Here’s an example of doing just that, in a single Scriptable Task element.

Instead of dropping a copy of the Action you created into the workflow, drop in a Scriptable Task.

Using a scriptable task instead of your Action.
Using a scriptable task instead of your Action.

These elements are generally used for specific tasks that are unique to the workflow.
They can also be the only thing a workflow designer uses if they are particularly comfortable and familiar with the APIs they are working with.

Once you have added the Scriptable Task element, edit it as-is. Input the following lines of code:

var output = System.getModule("a.nsfv.lab.repository").combineTwoParameters("Jump To","Conclusions Mat")

This format should be fairly simple to follow, but let’s break it down.
Remember that module you made at the beginning of the post?  This is where it comes into play.

You are defining a variable, output, and calling the Action combineTwoParameters which resides inside the module a.nsfv.lab.repository. When you call it, you are assigning the same string values you did earlier to the arguments in parentheses.

(Hopefully, that made some sense. I’m still working on my delivery.)

An interesting side note: if you do drag and drop the Action into your workflow and attempt to edit it, click on the Scripting tab. In there, you will see a line of code which looks just like the above. So the Action you are dragging into your workflow schema is really just an automatically generated script that calls an instance of your Action code.

How to call your Action code programatically, exposed!
How to call your Action code programatically, exposed!

I promise it’s not as complicated as it sounds. If anything, it is reducing the amount of code you would otherwise write.

Theoretically, you could call countless Actions in a single Scriptable Task. But you may not necessarily want to. The decision is up to you, but my advice would be to do as much as you can in the GUI so that others can more easily follow your work if they need to visit it. Not only that, I find it helps me out too, when I’m working on complicated workflows. It’s all a matter of preference.

My next post will continue with some simple VM tasks and working with the API Explorer. In the meantime, practice up, and look at the Actions that are already available to you. You may find that there are quite a few gaps – hopefully after going through these posts, you’ll be able to fill in your own gaps!