The Backup Blog

The Cost of Backup

I am going to revisit the issue of backup, cost, and the total cost of ownership of backup in this post. Tony Pearson and I have been discussing this (hopefully in a fairly constructive fashion) over the last little while, and I see he has responded to some of the points that I made in my previous post on the issue. Tony's response can be found here: "The TCO of TSM for Backup".

Tony continues to maintain (I think) that TSM has a "good" TCO and is efficient relative to other backup applications. Although I don't agree with a lot of what he writes, I do think that there are some pretty good lessons to be learned amidst the information he has posted, so I am going to continue to pursue this a bit in this post.

First, Tony claims that I said "only the cost of new TSM servers should be considered in any comparison." Ummm, jeepers Tony, no I didn't! I would never make any such claim! (What I did say is that TSM is unique in the backup world, in that it scales by adding master servers. Once my TSM master server is at capacity, due to I/O requirements, or database scale, or ability to drive disk or tape, I have to buy another master. That stands in sharp contrast to Networker or Netbackup, where I can just buy a Storage Node or Media Server--which tend to be much smaller systems with much less disk than the master servers. So TSM requires many more master servers than competitive solutions--each of which imposes not just a burden of cost, but a management burden at both the server level and the backup application level. Further, based on my experience, the typical TSM customer will end up with more master servers than they would have combined master and media servers if they were using Netbackup or Networker.)

Servers are but one of many cost considerations when it comes to understanding the TCO of a backup environment. Generally speaking, these costs can be divided into hard and soft costs. Hard costs are costs associated with physical, tangible things (servers, network, disk, tape drives, shipping, etc.). Soft costs are those costs associated with stuff that is less tangible. Soft costs can include labor and administration--though truthfully admin costs straddle the line between hard and soft costs, at least in part because they can be difficult to accurately quantify--but also include things like risk, efficiency, and the benefit of systems uptime.

DL3D Adds OST Support

This one sort of snuck out the door: with the latest version of the DL3D code, EMC has added support for Symantec's OST initiative.

OST is a way for Symantec NetBackup to address "disk as disk" rather than as virtual tape. More important than this, in my mind, is the ability of an OST enabled device to replicate in a much more intelligent way, and in a way that tightly integrates with the backup application.

What do you need from Symantec? Well you need NetBackup v6.5. And you need the OST plug-in installed on any media server that is going to connect to the DL3D. Connection over IP (only) is supported. And in Symantec's parlance, the DL3D becomes a "storage server".

This storage server contains Logical Storage Units which are available to Media Servers. The Media Servers then access the pool of deduplicated storage on the DL3D as they would any other NFS or CIFS attached storage.

NetBackup Embedded Media Server and EDL

One of the nifty features of the EMC's EDL is it's ability to run a NetBackup Media Server on the DL Engine. Just to back up a second: an EDL has 3 logical components: the DL Engine, the Clariion, and the disk. The DL Engine is where the good stuff happens that virtualizes disk as tape. It is also where data compression happens, system management occurs here, and so on. A couple of years ago, we enabled this server to run a full copy of either a NetBackup Media Server, or a NetWorker Storage Node.

Why? Because this would permit the EDL appliance to make a second copy of a virtual tape (either local or remote) and it would update the backup application catalog to show that second copy. No more did you need to worry about creating a replica with the appliance that was invisible to the backup application, and shared the same bar code (this is bad).

So life is good, right? Well, pretty much, but there is one little trick that we need to do to fully take advantage of the capabilities of the Embedded Media Server (EMS). And that is to utilize a feature in NBU called the "alternate read server".

Data Protection Advisor

Note: 2nd of 3 updates on new product announcements for this week. The 3rd installment will cover what is new with NetWorker 7.5.

I thought I would take the opportunity in the second of my product update discussion to discuss one of the lesser know, but most relevant and useful products in the lineup: Data Protection Advisor (formerly known as EMC Backup Advisor, and prior to that as WysDM). Data Protection Advisor, or DPA, fills a big gap for most backup and recovery administrators: the reporting, diagnostics, and analysis of their environment in a useful way.

Notice that I say useful: having 400 canned "reports" does nobody any good if the reports are not useful and relevant.

Deduplication and Replication Revisited

After my post two weeks ago, I wanted to come back to this topic and revisit some of the issues, largely thanks to a couple of very insightful comments from Curtis Preston. He raised some good suggestions with respect to replicating deduplicated data sets.

A brief recap of my prior post is this: how do you get the backup application to recognize the deduplicated data set at both the source and the target in such a way that you can use the data at both sites (for recoveries, data portability, data refreshes, etc.)? It isn't easy! I pointed out a couple of work-arounds, and Curtis added two. Because I think they are important, I am going to quote Curtis wholesale here.

Deduplication and Replication

Some time ago I wrote a post Peace, Love and Revolution about the issues that replicating deduplication appliances causes with backup applications. Some six months later, I thought it would be good to update the state of the nation and see what progress has been made.

To quickly refresh, the issue is this: how do you restore data from a deduplication appliance that is the target for replication--when it is the appliances that is managing the replication process and not the backup application. (And to be fair, this is not an issue unique to deduplication: the exact same problems apply if you are just using virtual tape. The issue just happens to be more pressing with deduplication because if it is "just" virtual tape, I can use the backup application to manage replication and not cost myself any bandwidth; with deduplication, if I use the backup application I miss out on the 95% bandwidth savings that deduplication replication promises.)

So They Do Understand...

It took a while, but the nice folks over at NetApp have attempted to describe what exactly Tape Smart Sizing does. I could be wrong, but I think they agree with me. I guess since I agreed with them that using single parity RAID on a VTL with 1 TB drives is dangerous (yes their Near Store VTL does exactly that...) they decided to return the favor? Perhaps...

In any event, Absolute Architect decided to pen a response which was completely complementary to my analysis. To the best of my understanding, she agrees that Tape Smart Sizing is a "confusing ... technology that won't remotely impact your life" (her words). Which pretty closely echoes my claim that it is "unnecessary and irrelevant". And I mean that literally. For most, if not all organizations, the "feature" is either unnecessary, irrelevant, or both.

Much like a square wheel bike.

Do square wheels work? Yes, if the ground is shaped like an inverted catenary. Are square wheels necessary or relevant? Not even remotely.

The analogy is perfect. Does Tape Smart sizing work? Yes. Is Tape Smart Sizing necessary or relevant? Not even remotely.

The good stuff is always in the why.

Tape Smart Sizing: Another NetApp Myth

One of the more interesting replies to my recent discussion of NetApp's dangerous VTL--which NetApp apparently feels is likely to cause significant loss of backup data--concerned the concept of "tape smart sizing". To quote NetApp's CTO: "I can also confirm that Tape Smart Sizing works exactly as advertised."

Since I dismissed it as little more than a David Blaine type trick, it might be useful to ask: what exactly is advertised? Again, just quoting NetApp:

"[tape smart sizing] sample[s] the data for compressibility as it’s being written to the VTL and then dynamically adjust the size of that virtual tape to match the estimated compressed capacity of the physical tape it will be written to. The LTO-3 virtual tape is now automatically adjusted depending on what type of data is being written to it and how it will compress in the Real-Tape World. Welcome to not wasting money or space on extra physical tapes when using a VTL."

Incidentally, NetApp also claims that:

Every other VTL out there requires you to to set a fixed size on the virtual tape when configuring the virtual library.

The consequence of this is, according to NetApp, that if you write 200 GB data that is 2:1 compressible to any other VTL, you will use 200 GB of disk space; write it to tape, and you use 100 GB of tape (because tape is compressed). So, it seems that you would require 2 virtual LTO-1 cartridges. Take those 2 virtual cartridges, and they will each half fill a physical cartridge. Thus, wasted tape.

So we have two different bloggers claiming that tape smart sizing is useful, and unique in terms of its ability to save space of physical media. Basically, that "every other VTL" will require that you waste physical tape space as described above.

Let me state this unequivocally, and for the record: this is not true. EMC's virtual tape products have no such issue.

News Bits

Time for a brief round-up of some of the news from the world of backup and archive.

First up, news from Symantec this week that they have bought SwapDrive so that they can offer a service like Mozy. This is a great validation of the direction the market is headed for small office and home user backup. I would offer the simple observation however that the service is much more expensive than Mozy--it ranges from $4.95 per month for 2 GB of backup data (which Mozy offers for free), to $44.95 per month for 50 GB (which Mozy offers for $4.95). So it is about 10 times more expensive, minimum. I protect close to 200 GB with Mozy which I assume would cost me a whopping $179.95 per month with SwapDrive's backup.com, as compared to $4.95 with Mozy. Nearly 40 times more expensive. They also don't appear to support Mac OS X at the moment, which would be another deal killer for me.

Peace, Love and Revolution

Barry Burke may want to riff on Britney Spears in his latest post, but I think I will channel Lucinda Williams instead. Besides being two or three decades older than Ms. Spears, Lucinda is also gifted with wisdom and passion, qualities sadly lacking in little Miss Train-Wreck-In-Slow-Motion-Spears. So, it is Lucinda's words that title this post.

And, I like to think that it is time for a little Peace, Love and Revolution in the world of backup and recovery. Mostly revolution though. The peace and love can come as we, as an industry, work to make things better.

The revolution we need is a re-think of our backup applications.