Veeam 12.1 – Dell Isilon SmartConnect

So this post is for letting you all know how to add the Dell Isilon (9.1) as a source storagesystem for your NAS backups WITH the use of SmartConnect.

It involves a few specific steps that might not be as straightforward if you are not a hardcore Isilon administrator (which I am not) 🙂

So to begin, step 1:
Make sure you have set-up the SmartConnect references on the Isilon. This is explained in more detail here: https://www.dell.com/support/manuals/en-us/isilon-onefs/ifs_pub_onefs_9100_administration_guide_gui/smartconnect-zones-and-aliases?guid=guid-0169dad0-b19f-4657-8b13-611a41cc5b63&lang=en-us

Go to Cluster Management -> Network configuration

Go to subnet0 and click “View/Edit” -> click Edit. Fill in the SmartConnect service IPs (this can be a range or just one IP). In my case this is just one IP. Also fill in the service name.

Go to Pool0 and click “View/Edit”, check the IP range and interface members. Fill in the “SmartConnect basic” info (zone name and service subnet), SmartConnect Advanced settings are also possible to set (if you want to, or just accept the defaults).

As you can see, if you look at ifconfig the Isilon nodes, is that one of them should have the Smartconnect Service IP attached to it.


Now that the Isilon has been set-up it is time to make some settings in your DNS server (in my case this is a Windows 2022 DC with DNS).

Start with adding an NS (Name Server) entry to the DNS that points to the SmartConnect Service IP.

Right click the domain where the Isilon is located and select “New Delegation”


Click Next

Fill in the delegated domain (in my case just isilon). Click Next.

Fill in the FQDN and NS record of the SmartConnect service IP.


After that you will (hopefully) get a message that the Delegation has been succesfully completed.


To check if it is working you can ping the isilon on the name you added to DNS (in my case just isilon). It will answer with an reply from one of the IP’s that are defined on the external range in the Isilon (in my case 192.168.10.101-103).


Now that this has been done and verified, the Isilon can be added to the Veeam infrastructure based on its SmartConnect DNS name, and for all its data actions in the future, it will leverage one of the ips from the external range (based on the policy that was set in the Isilon).

Go to “Storage Infrastructure” and click “Add Storage”.


Fill in the Isilon FQDN name of the SmartConnect that was added to the DNS server.


Select the Credentials to be used to access the Isilon.

Select the protocols to be used and click Apply.

Click Finish.

After pressing Finish a new “Storage Discovery” will be initiated. Click Close.


Now that it has been added to the Storage Infrastructure we can add it as a “Unstructured Data source” for the NAS backups.
Goto “Inventory” and select “Unstructured Data”, select “NAS Filer”.


Select the Isilon NAS Filer that was added to the Storage Infrastructure.


Select the “Cache Repository” and your “Backup I/O control” preference. For Isilon, you can use the “Native changed file tracking” and hit “Apply”.


Click Finish.


Now right click the Isilon and select “Add to backup job” and follow the wizard!


Your NAS job will be processed based on the SmartConnect settings that have been applied earlier. Happy times 🙂

Veeam Agent for Windows v6 – Direct Object Storage

It has been a while since my last post but I finally found some time to write a new one. This time it will be about Veeam Agent for Windows v6 (which is beta) with backups going directly to Object Storage (yes! agents that will directly backup to S3 without the need for anything else).

When configuring a job you will get the same questions as with previous versions of the agent. So give it a name.

Select what you want to backup, in my case my complete laptop.

Now for the destination, you can select to use “Object Storage”. This can be either on-premise S3-compatible object storage OR cloud object storage.

In my case, I select S3 Compatible because I will be using Wasabi.

Give in the specifics to connect to the object storage like “service point URL”, “Region” and of course “Access key and Secret key”.

Now….this part is very important! You can even select to use Immutability (S3 Object Lock).

Continue the wizard with the general questions like the “backup cache”.

Select if you want to use “Application-aware processing” and/or “File system indexing”.

And of course schedule your backups 🙂

As the final step select to “run the job when I click finish” and click finish.
Now you will see that the backup will go directly to Object Storage!!!

How to set-up gMSA with a secure One-way trust Veeam design in v12

As mentioned earlier in my post about how to set up a one-way trust Veeam design, I ended the post by mentioning gMSA (Group Managed Service Account). In this post, I will show how to use this more secure feature within a one-way trust Veeam design based on v12.

So to recap: I have a one-way trust set up between my veeamdemo.nl production domain and my veeambackup.nl backup domain, as illustrated in this sketch.

Configure for gMSA

To use gMSA, you will have to first add a root key in each forest where you want to utilize it.
This is done by running this PowerShell command:

Add-KdsRootKey -EffectiveImmediately

After running the command you need to add all machines that VBR processes for backup AND the servers used as “Guest Interaction Proxies” (for application-aware processing) to a security group in the production domain. In my example I will be backing up the MSEDGEWIN10 machine AND my Guest Interaction Proxy is called FS01.

Now it is time to actually create the Group Managed Service Account by running the following command on the Domain Controller in the production domain:

New-ADServiceAccount -Name gMSA-VBRv12 -PrincipalsAllowedToRetrieveManagedPassword SG-gMSA-Hosts

In Active Directory you should see the defined account added to the “Managed Service Accounts” group.

Now to test if the gMSA-account is effective on the machines that are in the Security Group you can run the following command on them and it should return a “True” value.

Test-ADServiceAccount gMSA-VBRv12

Now the gMSA needs to be added to the “local administrators” group on the machine you want to backup. In my case the MSEDGEWIN10.

Configure VBR to use gMSA

To leverage the gMSA from VBR we need to add it to the account list by going to “Manage Credentials” and fill in the domainname\gMSA-account

Now finally, in the job where you want to use gMSA you need to select the appropriate “Guest OS Credentials” AND you need to select the correct Guest Interaction Proxy that is living in the production domain.

Now click the “Test Now” button to see if it all works as expected 🙂

After that, you are good to go with using gMSA in your secure Veeam one-way trust design 🙂

How to update VBAv3 to v4

Getting the Veeam Backup for Azure v3 appliance updated to v4 can be a little bit troublesome.

First off all if your VBA is connected to VBR it needs to be updated from there. To do this make sure you have your VBR updated with the latest patches https://www.veeam.com/kb4245 (in my case, at this moment, that is: patch 20220302).

After applying the latest patch(es) it is time to download the latest Azure plug-in for VBR at: https://www.veeam.com/download_add_packs/vmware-esx-backup/azureplugin/

After downloading it has to be installed on the VBR server.

After installing the latest pacthes and plugin you will get an update window in VBR when you start the console.
Select the VBA appliance to update and click apply.

The update process can take while, so be patient.

After the update process when entering the VBA console you will be greeted by the new UI and first time splash screen and a small “What’s new in VBA” tour. Now enjoy your new VBA v4 🙂

How to make: a secure Veeam design based on a one-way trust

Making backups is, of course, always a good idea. If you store them offline or immutable, that’s even better 🙂 But as always, much more can be done to ensure you are as much protected as possible (or at least your backups are).
One of those things is to set up a backup resource domain with a one-way trust to your production domain where all the actual workloads live.
This way, you can make sure that only a small controlled amount of users have access to the actual backup domain and, by design, have access to your production domain.
Because there will only be a few users in the backup resource domain, you can really tighten the security on that part. That way, the attack factor can be as small as possible.

What would this look like?

Below I have made a small sketch of what this looks like from an infrastructure perspective and how the data flow would be travelling.

How to set it up?

So after the (simplified) design it is time to set it up. In this paragraph, I will try to show it step by step.

First step is ofcourse to have 2 domains in place. At the resource domain, go to “AD Domains and Trusts” and right-click the resource domain and select properties, select the “Trusts” tab and select “New Trust”.

Click Next and fill in the the production domain name (in my case veeamdemo.nl).

Select what kind of transistive trust it should be and select the one that is needed by your orginization. In my case I am choosing an “External Trust”, which makes it more secure by bounding it to the specific 2 domains in the trust (for a smaller attack footprint).

Select what kind of direction you want to use. In this case, because the “New Trust” wizard is initiated on the resource domain it will be a “One-way: incoming” trust.

In the next step you can tell it to create the trust on both domains (if you have appropriate credentials available for the domains, which I have).

After filling in the required credentials you will get the question to select the “Authentication Level”. In my case, I selected “Selective authentication” because I want to be totally in control of granting access to resources in the domain.

After this step, you will get an overview of the settings. Click next on this page.

After this the trust is created and can be verified (confirmed).

So after all these steps you should have a validated One-way trust between your two domains and making backups from a more secure perspective can begin.

Running a first backup job with the One-Way trust in place

Before we can start using VBR (which is located in the resource domain) to make backups of the production machines we will have to configure some settings in VBR.
First off, we will need to add a user’s credentials from the resource domain that you want to use for backing-up machines with AAIP in the production domain. In my case, I will be using my veeambackup\demo user.

For adding the vSphere infrastructure credentials need to be added too.
In my case I added the local administrator@vsphere.local account to see all the VM’s from the production domain (my advice is to use a specific local vSphere account instead of the administrator account 😉 and make sure your vSphere environment is not completely integrated and dependent on your production domain).

Now it is time to add a backup job to VBR. Start the new backup job wizard.

Select the destination for your backups and click next.

Now the most important step is the “Guest Processing” tab. Here you will have to select the right account to use for Application-Aware Processing. Here I have selected the “Guest OS credentials” based on my resource domain (veeambackup\demo).

Click the “Test Now” button to validate that you can access and communicate with the VM based on the resource domain credentials provided.

If all goes well, it should successfully finish the test. Go on with the wizard and complete the remaining steps.
Now it is time to run the actual backup.

As you can see, we can back up the machine with AAIP living in the production domain with a user from the resource domain.

So what else can we do to tighten security even more?

Well we can even use gMSA (Group Managed Service Accounts) so we can leave the password/user handling completely over by Active Directory (which will auto rotate and hand out passwords based on 240 bytes without us having to worry about it 🙂 ). I will discuss this in my next post.

Instant Restore SQL database to another server

This post is about instantly restoring an SQL database to a server in another domain/workgroup. Because of this cross-network instant restore, it is important to make sure you have adequate rights and permissions in the source environment and destination environment.

First of all, we start by opening the backup and tell it we want to restore to the point in time of the selected image-level backup. In my case, I am abusing the VeeamONE database and virtual machine for this exercise 🙂

Now for the specifying the target we need to fill in some details. Be aware that it must be possible to connect to the target on an FQDN so DNS should be working properly (might even need to add DNS suffixes on the interfaces).

DNS suffix added to the interface of the server that is performing the restore (my VBR-server)
ping the other SQL server from VBR
Target SQL Server connection parameters filled with FQDN, DB-name and credentials to connect

We must also specify a Windows user account to connect to the target server.

After this, it should either connect already and check if the credentials are in order or give an error message immediately that something is wrong.

The next step is to ensure that the paths used are correct (in my case I added “-Restored” on the end so I can easily identify it).

After this, you will need to select the database switchover scheduling option.

Step 6. Specify Switchover Option

The Instant Restore will now start.

To see that it is working, you can look at the Instant Recovery Session and see that it is copying target files, publishing database, etc.

But what you can also see is that the target server gets iSCSI initiator connections:

If you take a look at the target server and start the SQL Management Studio you will see that the database is being added.

The last step then is to do the switchover at a convenient time 🙂

Veeam Backup for Azure v2

With the release of Veeam V11 there is also the release of Veeam Backup for Azure v2 and Veeam Backup for Google Cloud Platform (see this blogpost). With VBAv2 there are additional features that can be used and the integration with VBR goes a step further.

How to integrate an existing VBAv2 appliance and how to use it?

That’s pretty simple and I’ll show it in a few simple basic steps.
First of all, from the VBR console go to “Backup Infrastructure” and right-click “Managed Servers”, select “Add server”. You will get the following screen in which you can select “Veeam Backup for Microsoft Azure”.

Select to “Connect to an existing appliance”.

Select your Microsoft Azure compute account.

Select the Subscription and the Data center.

Select the existing Veeam Backup for Azure VM (in my case VBA02).

Select the “Connection type” (in my case I have a Site-to-Site VPN to my Azure environment so I choose “Private Network”).

Select the “Credentials”.

Select the “Repository” and make sure that specific settings have been filled in like: “Credentials” and/or “Encryption Password” for the Repository.

Click “Apply” and VBR will do the rest and make the connection and add it to the console 🙂

So now: how to use it?

After the VBAv2 has been added to the VBR console we can see it as a “Managed Server” on the “Backup Infrastructure” page. What we can also see is that if the VBAv2 has already made backups and/or snapshots of Azure VM’s we see the section “Snapshots” and “External Repository” on the “Home” page.
From here on we can restore Azure VM backups with lots of options. We can recover it “Instantly” to VMware/Hyper-V or we can even restore it to AWS or another Azure tenant.
Many options that make “Cloud Mobility” and gives you the power and confidence to go from cloud to on-prem or vice versa or even from cloud to cloud, all without complications!

From the VBR console we can now also choose to make a new backup job for “Microsoft Azure”. In that case it opens up the VBAv2 webinterface on the page where you can create a VBAv2 backup policy.

All in all good improvements for VBAv2!

VBR v11 Linux Immutability and XFS – Gamechanger?

I wanted to write about this for a while now since I started testing and using the VBR v11 Beta, but now I finally can 🙂
Starting v11 of Veeam Backup & Recovery there will be a new feature that looks like it can be quite the gamechanger!
VBR v11 introduces Linux Immutability! Combine this with Linux XFS and you got yourself a very nice and efficient repository to store backups knowing it cannot be altered until the defined threshold time passes. If you have a Public Cloud to your disposal (for instance Azure) you can even deploy the Linux XFS repository in the Cloud so it is stored off-site even.

In this blog I want to show you how to set it up and demonstrate it really works. So here goes!

First of all, I deployed an Azure VM running Ubuntu 20.04 and set up XFS on a additional Azure Data Disk (1024GB) Premium SSD.

After deployment open up SSH to the machine and set up the XFS data disk.

Run the following commands (in my example /dev/sda is the Azure Data disk of 1024GB)
sudo fdisk -l /dev/sda
sudo fdisk /dev/sda
answer N
answer P (for primary)
answer 1 (for partition number)
press ENTER (to accept default first sector)
press ENTER (for complete disk size)
answer W (for writing partition table)

Run the following volume format string command to configure a Linux backup repository for work with Fast Clone:
https://helpcenter.veeam.com/docs/backup/vsphere/backup_repository_block_cloning.html?ver=100
sudo mkfs.xfs -b size=4096 -m reflink=1,crc=1 /dev/sda1

Create a directory called “backups” (for example) and make sure to mount the drive during boot to it.
sudo mkdir /backups
sudo blkid /dev/sda1
su
sudo echo ‘UUID=uuid-from-blkid /backups xfs defaults 1 1′ >> /etc/fstab (you can ofcourse also use VI or nano to edit /etc/fstab)
sudo mount -a
df -h | grep /backups

Now XFS has been configured and is almost ready for use. We will create a user first with no rights and make him owner of the directory so we can add the repository to Veeam.

sudo useradd -m veeamrepo
sudo passwd veeamrepo
sudo chown -R veeamrepo:veeamrepo /backups
ls -alh /backups
For adding it to Veeam we need to give it sudo rights temporarily.
sudo vi /etc/sudoers
veeamrepo ALL=(ALL:ALL) ALL

Next step is to add this Linux Repository to VBR v11 to make use of it. Within VBR v11 we will be setting the Immutability option when we configure the repository.
For adding a Backup Repository you go to: “Backup Infrastructure” > “Backup Repositories” > “Right clik and select Add Backup Repository” > “Direct attached storage” > “Linux”. There you give it a name and click “Next”.
Now from this point you select “Add new” and fill in the required information.

Use the veeamrepo we created with the new option: “Single-user credentials for hardened repository”

Select “Yes” for the fingerprint message to trust the server.
Now VBR will start updating the infrastructure with the new information and do some checks to see if the Veeam Data Mover service is already there or needs to be installed and configured. After all checks have passed we can continue.

At the “New Backup Repository” screen we can now “Populate” it to find all the available Paths. Select the created XFS drive in my case /backups (/dev/sda1).

Select the “fast cloning” option for the repository AND also select the “Immutability” option (I set it for 7 days).

For the “Mount Server” and “Review” I left the defaults in my situation.

If we apply then al settings will be checked and I all goes well all lights turn to Green !

Now we have to remove the sudo rights from teh veeamrepo to make sure it is really hardened!
sudo visudo /etc/sudoers and remove the veeamrepo ALL line !

Now further harden the the Linux Repository by also disable SSH with the following commands:
sudo systemctl disable ssh
sudo systemctl stop ssh


Now we are ready to configure/add a backup job which uses our new “Linux XFS Repository with Immutability option”. Make sure to use Incremental with Synthetic Full option in the Advanced options for the job.

After this, we can take full use of XFS and Immutability on Linux!

Testing the Immutability

First we select the backup that is stored on the new XFS-Immutable repository and select “delete from disk”.

Veeam will try to delete the backup from disk but will give us a message:

As you can see it failed to delete the backup because of Immutability.
If we take a closer look at the Linux XFS Repository itself we can see the following if we try to delete it from there:


So now we have created a Linux XFS repository with the Immutability option from VBR v11 which allows us to secure backups for a certain amount of pre-defined time, this way ransomware has no change to compromise the backups.

Veeam Backup for Office365 v5 !!

So I finally had some time to make a post about Veeam Backup for Office365 v5. Currently, it is still in beta but it has been shown on Veeam Live already!
v5 has some very interesting features, with probably the most noticeable eye-catching one being: support for MS Teams backup AND restore. And you might be wondering now: why would this be so great news…well Veeam is, in v5, doing it from an MS Teams perspective. And that fact makes it such a huge feature.

To jump right in I assume you know VBOv4 and how it works and today I want to show the difference that v5 brings from the MS Teams point of view because as everyone knows MS Teams has taken a huge leap of adoption among customers and it seems that everyone is using it these days one way or another.

So what does VBO v5 bring for MS Teams? Well first of all it has become a selectable part of the product. In v5 you can select the option that you want to specifically backup: MS Teams.

After the backup job has run you can start MS Teams restores by starting a restore operation which will start our famous Veeam Explorer and you’ll get the following (based on your tenant):

Posts can be restored to HTML, Files can be restored to their original location or a location of your choosing. Channels and Teams themselves can of course be restored, and all restore options together in VBO v5 bring it to some 40 restore options (for Mail, SharePoint, OneDrive, Teams)!