14 Oct Offloading VST’s to Network for a Hardware Agnostic Setup?
So last week, I came up with a “smallish” project for my studio. Some of the results did turn out interesting and as a result, thought that writing on the subject for others might be a good idea.
A couple of years back, I made the decision to migrate my entire studio rig over from Mac to PC (for a couple of reasons I wont outline here). Even though it hasn’t been all sunshine and rainbows, for the most part, everything still works as it should. However, I suddenly realized….holy crap! I’ve set myself up for disaster!
Over the years, I’ve built up a pretty large sample library (~2TB) and many of these libraries are attached to plugins called VST’s which also are attached to different licensing keys and authentication methods such as iLok, the Steinberg Key, the Vienna Key, or just plain old server response authentication.
Here’s the conundrum.
What happens to all of this if say you needed to refresh your system or completely reinstall your operating system due to malicious software masquerading as virus protection software and you mistakenly give that software elevated privileges to do what it wants to “fix” that virus it detected? Or how about unintentionally installing an OS upgrade that inevitably makes your system slow and you need to roll back everything, or even worse, you brick your entire system prompting for a reinstall of everything?
But even with all those admittedly terrible scenarios aside, what if you simply just wanted to change your OS platform from Mac to PC (or Linux >_>), or you just wanted the flexibility to switch between either platform while keeping your authentications in tact and your plugins and sound libraries in one place? It’s a REAL hassle to go through ALL OF THOSE INSTALLATION programs all over again.
So the project became, “I wanted to create a system where I could make my plugin installs hardware and maybe even OS agnostic”.
Here’s the approach
Within my network, I have a file server and a vm server that I use daily. Since the file server is a much more robust storage solution, I figured that I would move my entire 2TB library over to that storage container instead. Then on the local system, instead of running the plugin installations again to give it the new paths, I would simply create a symbolic link on the local machine to the network share where the plugins are hosted.
mklink /D “[OLD FOLDER LOCATION]” “[NEW FOLDER LOCATION]”
That would take care of the actual core files for the libraries, but what about the actual plugins themselves?
Fortunately there’s a software by Vienna Instruments called Vienna Ensemble Pro that I purchased years ago for another project. I didn’t use the software because my needs for it changed, but it was nice that I still had it available. If I was going to go full throttle on this idea though, I would have to upgrade my version to the latest which thankfully, isn’t that much.
Here’s the breakdown of the VEP idea
The idea of using VEP would be that the VM machine would become the plugin host and serve as the single authenticated point for all my plugins. This would mean that all the local machines would have to do is pull up the VEP client plugin to request whatever plugin from the VM machine…basically, the production machine would kind of “act” as a “thin client” where the plugin server does all the heavy lifting.
What this would also mean is that if for whatever reason the local system has to change platforms, or an OS needed to be reinstalled, the only plugin that would need to be reinstalled is VEP as all the authenticated plugins are safely on another system! I don’t know about you, but that’s MUCH easier than reinstalling every single plugin by hand on the local system.
So how did this all turn out; seems like a solid win all around yes?
I mean, the initial realization was that in fact yes, I would be introducing a bottleneck on sample loads because now it’s up to your network to serve those samples down every time. In context, I did recently redo my network stack though, but even though I do have CAT6a wiring, the network is still basically a 1Gbs network (though the server has an aggregated 4Gbs pipe). Thing is, even with the bottleneck in play, it would only be at the initial load of each sample since once loaded, it’s not like the DAW calls back to the hard drive anyway since all those samples are loaded into RAM.
So it should be fine right?!
…BOY WAS I WRONG!
To be clear, linking your library using a sym link to a network share this way DOES work! I did not get any weird errors about a library not being found or anything. That’s because a sym link is treated as if it is the actual data. It’s not like a regular shortcut link. The problem is that the initial sample loads were UNBEARABLY SLOW!
Just to load up an 18 piece violin set from PLAY took nearly 10 MINUTES where locally, that shouldn’t take more than 2 minutes to load! Of course once the samples were loaded, the project played perfectly, but imagine if you did this for an even larger orchestra piece were you have like 20 or 30+ of these kind of sets in one project? You’d be waiting forever!
Unfortunately since putting the libraries on a network share wasn’t ideal at all, there would be no way that the plugin server would be able to get the library files to serve without me having to completely install everything on the server machine locally which wasn’t going to be ideal either and sort of defeats the purpose of using a robust storage solution in the first place. So VEP may have worked, but due to the former part of the project failing, I have not tested it.
But there is some light at the end of this project
Fortunately I realized that I completely forgot that I had been doing a silent differential backup of my entire sample library to the file server!
These differential backups (made with Macrium Reflect) are then pushed to an offsite backup solution using Crashplan. So I essentially have three instances of the data available to me; on the local studio machine, on my file server, and on an offsite server. While this unfortunately does nothing for making the installs hardware and OS agnostic, this does prevent one of the largest pain points of those installs should I need to do them in the future; loading in the core library files. Backing up the non iLok and Steinberg keys and authentication methods would also need to be reconsidered, but I digress.
Going forward, I may just look into changing over the storage platter drive for my sample libraries to an SSD or maybe some sort of SSD RAID array, but for now, I’m just glad that I do have my data for the most part secured and available when I need it, or in the worst case, if there’s an emergency where I need to recover it.
So let’s review
- You CAN use a Symbolic Link as your plugin directory to another location since sym links are treated as the actual location of the data unlike a shortcut.
- Unless you have a really fast network (Probably 10G), expect LONG load times of your samples if you plan to host large sample libraries on a network drive.
- BACKUP YOUR DATA
- You can use these tools to backup your data:
- For offloading heavy plugins from your studio computer and possibly introducing a hardware and os agnostic setup, you can use Vienna Ensemble Pro.
- Note: Just realize that you will need a separate Vienna Key for each system that you plan to use aka one for each client, and another for the server.
I hope this helps someone, or a few of my composer friends and if you have taken the plunge with Vienna, I’d be interested to hear your results!