The Backup Blog

Data Protection Advisor 5.5

EMC is releasing the latest version of Data Protection Advisor (DPA) today, version 5.5

DPA has been with EMC for a while now, and it is one of those quiet little overachievers that probably doesn't get the recognition it deserves. One big reason for that is the chronic perception of backup as the most unwanted, and unloved, discipline in the IT organization. And it can be hard enough for backup folks to get funding for infrastructure and software, never mind a piece of software that is (in some senses) secondary to the whole process: software that monitors and reports on backup.

For better or worse however, this stuff is important. Simply put, the native reporting capabilities of most backup applications is not good.

As a result, most backup admins--and their organizations--benefit a lot from software that helps them monitor, report, trouble shoot, do trend analysis and so on. What is maybe not clear is just how much this stuff can help: we have found that it is often the case that DPA is so helpful that it pays for itself in a year or less. DPA can save so much administrative time and effort that it cost justifiable in a very short time frame.

Do You Need Backup?

Do you need backup?

And yes that is a serious question. (Given the name of this blog, and my chosen specialization for the last 20 years, it is understandable if you think I was asking it in jest!)

A recent real world situation makes me wonder. It also makes me wonder at what point something becomes a self-evident truth? And are we adequately equipped to answer this question with a logical and defensible answer based on data rather than opinion? Hmmm...

First, a story. One of my favourites in years of IT. And it really happened, about 12 or so years ago.

A certain organization had a badly aging mainframe. It was creaking badly (in the metaphorical sense). The maintenance cost way more than the lease on a newer system. It was bigger than a newer system--taking up too much space. It wouldn't run the most current operating system. And so on. And the only one that noticed was the lead sys admin. When he pointed this out to management--who were apparently surprised by this state of affairs--he asked for funding for a new system. His manager asked him to justify the funding request. The sys admin left, and came in early the next morning to drop off a one page business case for the upgrade. Actually, it was just three words (in 72 point text): "Think about it."

Backup Cost Reduction, Target Deduplication, and TSM

Two posts previous I was discussing the TCO of backup, the differences between hard and soft costs, and the qualities of a good business case. The general conclusion? Most business cases depend on hard costs (and don't recognize soft costs). If hard costs show an ROI of less than 18 months, most CIO/CFO types will love it. If the business case relies on soft costs to get there, it will probably be rejected.

Looking at the particular example from the previous post, the business case Tony Pearson used as an example showed that 81% of the savings were due to soft costs. This is really bad. In most business environments, such a business case would be rejected out of hand.

I thought it might be instructive to have a look at a better business case: a business case based almost entirely on hard cost (savings). Two of them actually. One for source deduplication, one for target deduplication. In both cases, they are real environments, with real costs, real savings, the proposals were made with the costs and savings exactly as described, and they were accepted by the customer and the projects were funded on that basis. I have only removed names and specifics to the extent necessary to anonymize the customer and any technical or business characteristics unique to that customer. And, as I wrote the TCO tool itself EMC uses for target deduplication, I am more than happy to answer any questions on the data, the assumptions that drive it, or the calculations and formulas that we used to determine the benefits and cost savings of the solution.

In this post, I am going to deal with target deduplication. The business case for a source deduplication solution will be the subject of a future post.

Cloud Backup Event

I have been running a poll for a few months now that asks the question: would you back up your business data to the cloud? I have to admit that I find the results a little surprising, but they have been consistent over time, and are relatively consistent across geography. Approximately 50% of respondents have indicated that they are willing to backup up to the cloud.

With the latest release of NetWorker, EMC has introduced cloud backup as native functionality. Right out of the box, NetWorker users can clone backup data to a target in the cloud. EMC is hosting a Live Event that discusses the functionality in greater detail.

EMC NetWorker Fast Start and the new Cloud Backup option offer a secure and affordable alternative to the pain and expense of offsite tape vaulting. Attend this webcast and learn how you can:

• Reduce personnel expenses by automating offsite data protection
• Cut the hard costs associated with tape infrastructure and offsite storage facilities
• Choose a flexible “pay-as-you-go” model for offsite storage

You can register for the event here, or by going to http://info.emc.com/mk/get/DBM5396-1950_RAF_LP?reg_src=PA .

The Cost of Backup

I am going to revisit the issue of backup, cost, and the total cost of ownership of backup in this post. Tony Pearson and I have been discussing this (hopefully in a fairly constructive fashion) over the last little while, and I see he has responded to some of the points that I made in my previous post on the issue. Tony's response can be found here: "The TCO of TSM for Backup".

Tony continues to maintain (I think) that TSM has a "good" TCO and is efficient relative to other backup applications. Although I don't agree with a lot of what he writes, I do think that there are some pretty good lessons to be learned amidst the information he has posted, so I am going to continue to pursue this a bit in this post.

First, Tony claims that I said "only the cost of new TSM servers should be considered in any comparison." Ummm, jeepers Tony, no I didn't! I would never make any such claim! (What I did say is that TSM is unique in the backup world, in that it scales by adding master servers. Once my TSM master server is at capacity, due to I/O requirements, or database scale, or ability to drive disk or tape, I have to buy another master. That stands in sharp contrast to Networker or Netbackup, where I can just buy a Storage Node or Media Server--which tend to be much smaller systems with much less disk than the master servers. So TSM requires many more master servers than competitive solutions--each of which imposes not just a burden of cost, but a management burden at both the server level and the backup application level. Further, based on my experience, the typical TSM customer will end up with more master servers than they would have combined master and media servers if they were using Netbackup or Networker.)

Servers are but one of many cost considerations when it comes to understanding the TCO of a backup environment. Generally speaking, these costs can be divided into hard and soft costs. Hard costs are costs associated with physical, tangible things (servers, network, disk, tape drives, shipping, etc.). Soft costs are those costs associated with stuff that is less tangible. Soft costs can include labor and administration--though truthfully admin costs straddle the line between hard and soft costs, at least in part because they can be difficult to accurately quantify--but also include things like risk, efficiency, and the benefit of systems uptime.

TSM and the Elephant

OK, fair notice here, but much of what follows is only tangentially related to backup, at best. But there is a point, I think, to my rambling.

First, let me start by reacting to Tony Pearson's continued defence of TSM as a backup application that offers economies of scale. And before I really get going, let me emphasize again the definition of economies of scale: economies of scale offer a declining marginal cost per unit. It is what makes many industries go around: automotive (to the degree that it is actually going around these days!) and chip manufacturing to name just two. It is what makes Intel so successful: the 1,000,000 chip of a given architecture costs pennies (although the first chip might have cost them $2bn or so!).

I have a belief that is somebody can figure out how to achieve real economies of scale in backup and recovery, they are going to achieve two things: they are going to get rich, and they are going to get significant (dominant?) market share. Although I think that EMC has gone further than anybody else in achieving economies of scale with backup, I think there is still a great deal more that we can do. We as an industry, or we as EMC, however you wish to read that.

The Cost of Backup: Soft and Hard Costs

Continuing a theme from the previous post, I want to briefly discuss the notion of hard and soft costs in a business case or Total Cost of Ownership (TCO) study.

Hard costs are those costs associated with hardware, software, maintenance, operational expenses, capital costs, and so on. They are specific and identifiable costs to the business. Soft costs, on the other hand, are not so easy. They are often highly variable, based on intangibles, or things that are very difficult to quantify. They include things like the cost of downtime, the value of data, the benefit of higher availability, and sometimes, people and labor costs.

(Although, as an aside, I would note that people/labor costs can sometimes be soft costs, and sometimes be hard costs. If we are talking about the cost of personnel to administrate a backup system, this is a hard cost. If we are talking about lost productivity due to system/data unavailability, this is a soft cost.)

Generally speaking, most businesses will accept the distinction, and will accept hard costs and savings in a business case. It is pretty tough to argue that a tape library with tape drives carries a real and quantifiable cost: a cost of acquisition, and a cost of maintenance.

On the other hand, some businesses will accept soft costs, but many will not. Do you really save $1,200,000 if your mission critical server can be restored 4 hours faster than with a competitive solution? Do you really lose $10,000,000 if you lose (and cannot restore) 200 MB of data? Maybe. It probably depends on your industry. A financial institution may lose more, and have a better understanding of the magnitude of those losses, than some other businesses. Other businesses may not lose anything except people time. And, for better or worse, people time may not be valued with a specific dollar amount. So, for many, a business case based (solely) on soft costs is a much tougher sell.

The Remote Office: CDP or Backup?

A colleague sent me the following link the other day: The Case Against Tape Backup. The link gets you to a white paper written by Double Take Software, which argues that using tape for remote office backup is a bad idea.

Their conclusion? That instead of tape, we should employ CDP software, and that this is much more cost effective.

Unfortunately the paper is deeply flawed. Don't get me wrong, I would be very happy if the general conclusions they drew were true. I would be just as happy if they were content to conclude only that tape was bad--meaning too expensive and too unreliable. 

However, they are not content to reach such a conclusion, and in their attempt to make the leap from "tape is bad" to "CDP is good" they go dramatically awry in two respects. First, they posit a false dichotomy. It is not like the only alternative to tape is CDP for remote backup. And, in fact, some of the solutions that they (conveniently) ignore, are substantially less expensive than their proposed CDP solution. Second, their financial reasoning is pretty badly flawed. And I have never been a fan of make your own solution look good by falsely making the alternative solution look bad.

However, I do think that following through the line of reasoning in the white paper is pretty interesting--and by doing so we can expose some better data about the costs of remote office backup and the alternatives to tape.

Putting The Pieces Together: Deduplication Technologies

Disclaimer: everything that follows is my opinion. In no way should it be mistaken for any sort of official EMC position. Any resemblance to that is purely coincidental.

Amidst the chaos and confusion of the EMC offer to buy Data Domain, I think the biggest unanswered question has been: why would EMC want to have three or four or five different deduplication technologies?

Truthfully, I think the question has a profoundly simple answer: because backup sucks.

Mmmm, irony.

Don't believe me? Show of hands: how many people actually like their existing backup application?

Given the name of this blog, and that I have spent the last 15 years plus on backup and recovery, I think I can appreciate the irony as well as any.

Even more ironic when we remember that Data Domain's original slogan was: tape sucks.

And by the way, there is a back handed answer in there as to why it makes sense for Data Domain to be acquired by somebody. If you claim that tape sucks, and you try to "fix" this with another piece of hardware, all you are really doing is building a better tape drive. You are trying to be StorageTek, only better. Where better means faster and cheaper. But does a faster and cheaper tape drive make backup suck less? Maybe. But probably not nearly as much as it could--if you had a broader perspective and reach with your technology roadmap, one that includes CDP, primary storage, a backup application, and some virtualization capability. More on what I would do with all that later. But one final question: if your objective is to fix backup, completely, and you think that you need access to all those components to do that, who is going to be in a better position to do this? EMC? Or NetApp?

Having said that, the biggest obstacle to fixing backup is not technology. It is inertia. It is cultural. It is fear of change. It is ingrained process. It is the fact that we have done things one way for so long that the reason we are going things has been forgotten.

Backup Service Catalog

My previous post Is a Copy a Backup? was written after I got to thinking about a backup services catalog. What is a backup services catalog? Basically it is two things.

First, it is an admission that we have a lot more choices in terms of what technologies to use to provision backup services than ever before. 10 years ago, the backup technologies was really backup technology: tape. It was pretty easy to design a such a catalog, because you really only had one choice: tape. And it didn't really matter how rich, thin, or beautiful you were, you still only had one choice: tape.

Second, it is a recognition than different applications have different levels of importance to the business, and that given we have an array of technology choices with which to backup them up and recover them, we should probably choose different technologies for the different applications. Let me put that a different way: your SAP/Oracle application which runs AP, AR, and inventory is probably way more important than a miscellaneous SQL server some user threw up for ad hoc reporting, or an edge web server that is one of a few dozen others just like it.

So if you have different tiers of applications, shouldn't you have different tiers of backup and recovery? You bet you should.