In an effort to respond to the observations of W. Curtis Preston in his posts here, here and his comments here; and additionally in an effort to satisfy those of you who wanted a more direct response to those observations, I post some (final?!) comments below. Two comments first:
- The dialogue has been going on for a long time, and the material comes from a few different sources. So the responses are more point form, and may move from point to point without a lot of regard for the previous point--there is no longer any strong unifying theme.
- All the material below should, in my opinion, be placed firmly in the context of my VTL: What Matters? post. I feel quite strongly that those are the over-riding concerns, and that the following material is interesting primarily in terms of its' ability to determine how well the device meets the criteria described in "What Matters?"
With that said, on with the show!
1) Lets start with with the most incendiary comment. Curtis wrote: "It looks like it costs more and offers far less value than any of the alternatives, so I’m simply at a loss as to why someone would buy it over a competing solution."
Well, I have been pretty clear on this from the get go: the DL4000 exists for people that need to get a lot of data backed up in a short window. If you need (approximately) 8 TB an hour, and you need to deduplicate that data at some point, the DL4000 is a great choice. If you need to replicate that deduplicated data, it may be the only choice (bearing in mind that Sepaton and HP can not replicate deduplicated data, and nobody else is remotely as fast).
Now Curtis keeps insisting that you deduplicate within the backup window. Because he also insists that you have to finish your off-site process within that window. Which, if I can be blunt, is nonsense. Very few (no?) organizations both make a primary backup copy, and an off-site copy, and get it to an off-site storage facility within a backup window. Far more prevalent is one of two other scenarios: you make a backup within the window, give it to a courier and it goes off-site, and it is at the storage facility by sometime that afternoon. But your primary copy is gone, off-site, and isn't available for restore purposes unless you call the courier and ask for an emergency recall. The other scenario is that you make a copy of your on-site data to tape, and ship that off-site. But because all the tape drives necessary to finish that process by early morning are really expensive, people are more relaxed, and are willing to take business hours to finish. Meaning the off-site copy only goes off the next business day.
So lets put that back in DL4000 terms: I can back up at 8 TB/hr, and deduplicate at 3 TB/hr. Meaning my backup can finish within my window, and if my deduplication (and consequent replication) finishes much later--because it is only running at 40% of the speed of the initial backup--that is fine, because my SLA is still better than I had before with tape.
So, as I have said many times before: if you want to get a lot of data backed up, and you are serious about finishing that within a window, the DL4000 with deduplication is a great choice.
And that, incidentally, is one of
the core reasons why I wrote the "What Matters" piece. The speeds and
feeds are relevant insofar as they give (or not) the ability to meet a business
requirement: a backup window, or a SLA
As far as the cost component: again, lets be blunt: Curtis is just speculating. He is saying, basically, because it has parts x, y, and z, and the other vendors only have parts x and y, it must be more expensive. Well, based on my observation, the platform is price competitive with other large scale VTL platforms. Never mind what Curtis reckons it should cost, what does it cost?
So when Curtis asks why someone would buy it over a competing solution, I can answer: because it is faster than competitive solutions, offers deduplication and replication, is supported by a single vendor, and does so at a competitive price point.
2) Next point. Curtis writes: "It seems that there is a front-end and back-end VTL, and it seems that the way that data is moved between them exactly matches how I described it. ... More specifically that it is moved around more times with the 3D 4000 than any other dedupe system"
OK. But who cares? Seriously: who cares? Let's just stipulate it is move 120 times in one day. Why does that matter? It would only matter as much as it impacted general systems performance and availability. And given that our performance is so good, I am really not sure what Curtis' point is here?
3) Curtis writes: "Since data is coming in faster than the dedupe process can dedupe, though, it can’t get it all from RAM. ... That would suggest that a significant portion of the data to be deduped will be read from disk. I’m guessing at least 75%. ... All I’m saying that the additional I/O required by the 3D 4000 architecture isn’t free. It requires additional I/O paths and additional CPU and RAM resources to make it happen."
Again, same verse, just like the first: who cares? As long as the system is price competitive and no more complex to implement, manage, support, or maintain, who cares how many paths, CPUs, or resources are involved? And Curtis offers no evidence that any of this is case. Just speculation that it "must" cost more. Why? Well, it just has to, doesn't it? Doesn't it? ...
4) Curtis writes: "There appears to be more than “one copy of data in the system” and the possibility of having data in both places “always” (if you choose the option to never delete the copy on the EDL)."
With any post process deduplication system, there will be more than one copy of the data in the system for some period of time. His suggestion that data could be in both places "always" doesn't make any sense at all to me, unless your VTL is as big as your deduplication pool, times the deduplication ratio. Which doesn't really make any sense at all.
I have said in the past that the DL4000 maintains data fully hydrated for as long as you want. Which is true. I guess I didn't think I needed to also state: "or until you run out of capacity." So there is no possibility of having data in both places, unless you have a system of infinite capacity. This just makes no snse.
5) Curtis writes: "3D 4000 appliance does not support Path to Tape feature” and that the "3D 4000 does not support either support [sic] NAS backup or NAS sharing.”
Both of these seem to me to miss the point pretty significantly. We don't support path to tape or OST on the DL4000 because we have a superior methodology: we support Media Servers and Storage Nodes directly on the DL4000 DL Engine. Both offer superior functionality to path to tape.
Similarly, we don't support NAS backup for the trivial reason that this is a virtual tape library (and therefore, by definition, requires a SCSI/FC mount point) and for the much more signficant reason that NAS doesn't offer the performance we require. If you need 2,200 MB/s, you are not going to get there over Ethernet. Not yet. And just as importantly, if you need that kind of speed, I believe you are 99.99% likely to be currently using FC for your backup.
6) Curtis writes: "But other post-process dedupe systems (Exagrid, Quantum, SEPATON) are different. You can change your mind any time you want to as to how much data is stored in deduped format and how much is stored in native format."
Unfortunately, I think Curtis has things completely backwards here. The DL4000 allows you to choose how much data is native, and how much deduped (on the front end and back end, as Curtis refers to them) by policy. Assuming you acquire sufficient capacity, it is up to you to make the simple policy choice: how long will I leave data fully hydrated, in native format? And how long after backup will I deduplicate it? (With the answer to the second being anything from seconds to days...)
I am not sufficiently familiar with Exagrid to comment, but Quantum has no policy engine that can mimic such behaviour. And Sepaton only permits one day of data in the native format. As far as I know (and I could be wrong!) it dedups all data older than one day.
So Curtis has it backward: in fact the EMC system is the only system that has the flexibility of a policy engine, and there fore the ability to store data in native format for as long as you want, and deduplicate it when you want.
7) Curtis writes of the performance of a DL4000: "You can have two nodes on the front, but they must share a single node in the back if you're to get global dedupe. You can also put two dedupe nodes in the back, but they don't have global dedupe. So you get either 1100 or 2200 MB/s, but always 400 MB/s in the back. "
This is just not correct. You can have a system that does 1100 MB/s and 400 MB/s. Or you can have a system that does 2200 MB/s and 400 MB/s. Or you can have a system that does 2200 MB/s and 800 MB/s. Your choice.
8) Curtis writes of the DD690 that it offers performance of 750 MB/s on ingest and deduplication, and notes this is "approximately" 25% less if you don't have OST.
Acutally, it is 400 MB/s if you don't have OST AND 10 Gigabit Ethernet.
9) Curtis has the following chart performance metrics, where the first number is ingest speed, and the second is deduplication speed (all in MB/s):
- EMC: 1100 or 2200 / 400
- Data Domain: 750 / 750
- FalconStor/Sun 11000 / 3200
- IBM/Diligent 900 / 900
- NetApp 600 / unknown
- Quantum/Dell 880 / 500
- SEPATON/HP 3000 / 1500
And he offers this observation: What I see is twice as much storage for not much more (and often less) performance when compared to competitors."
First, what should the numbers be? They should be as follows (removing cases like the Falconstor box which, as even Curtis admits, can only exist on paper):
- EMC: 2200 / 800
- Data Domain: 400 / 400
- FalconStor/Sun: 5500 / 1600 (and I haven't seen any evidence of anything like that having been implemented in the real world)
- IBM/Diligent: 900 / 900
- NetApp: 600 / unknown
- Quantum/Dell: 880 / 500
- Sepaton/HP: 1200 / 1200
Now, lets eliminate the folks that can't replicate, like both Sepaton and HP. What we are left with is only one vendor that can come close the performance of a DL4000: Falconstor/Sun. And, no malice, I haven't seen any evidence of clusters of these actually being deployed. (And ignoring the fact that Sun may cease to exist, if current rumours of an IBM acquisition bear any truth.)
That leaves the DL4000 at approximately 3-4 times the ingest rate of any other competitor.
As far as it requiring twice as much storage? Well, any post process system is going to require native capacity and deduplication capacity. So EMC is no different than any other vendor in this respect. And therefore, Curtis is simply incorrect to say that we require "twice as much storage" as our competitors. We may require some additional storage as compared to in-line only deduplication vendors (where "some" is equal to your largest daily backup") but there are distinct benefits to this too--primarily backup and restore performance.
10) Curtis writes: You keep
saying that it's acceptable to let dedupe take 20 hours. I'll agree with that
IF the customer isn't replicating and/or that replicated copy isn't their first
copy offsite. Because with a 20-hour dedupe window ... you'll finish deduping
at 4 PM, after which the last of the backups will be replicated offsite. ...
That's 6-10 hours later than most people want to get their backups offsite,
compared to the Iron Mountain
Lastly, Curtis makes a fundamental mistake here: deduplication does not occur after deduplication, but simultaneously, on the DL4000 platform. The system replicates as it deduplicates. So ideally (and assuming your network can handle it), if you finish deduplicating at 4 PM, you finish replicating at 4 PM.
Considering my comments above about the only backups that are off-site by 9 AM are the backups where the only backup is off-site (vs. an EDL where you have both an on-site and off-site copy), I think we can safely conclude that if I finish my replication by 4 PM, and at that time I have an on-site copy and an-off site copy, that is a superior SLA to tape.
If that just isn't good enough: if you need to have your replication done by 9 AM, then you need to deduplicate in-line. Not only will a DL4000 not meet this requirement, neither will any other system from any other vendor that I am aware of.