After landing in Warsaw, I got into a car with the local sales leader for Poland and we drove to the event location.  It was a 2 hour drive.  First, the roads and the land in Poland reminded me very much of my home time in Maine.  Very scenic and rural but beautiful and peaceful.  We talked storage for 2 hours and I am always festinated at the thirst for knowledge there is when I travel.  It was a great ride followed up by a customer reception and some local Polish brew.

Thursday I spent the day in Sterdyn, Poland for IBM Storage University.  There were 30 customers at the event and it went very very well.  The event was at Palac Ossolinski, today used as an event center but has a very rich history, in fact at one point it was used as a medical facility in WWII.  The photo is of the building where we had the event.  The topics we covered were:

• Storage Efficiency
• EasyTier
• ProtecTIER
• XIV
• Real-time Compression
• V7000

The customers were very interactive and provided a lot of insight to their environments.  Interestingly enough I learned during our customer reception that IBM storage is #1 in Poland with HP second and EMC third.  This is a true testament to the IBM sellers and the customers who use the IBM products every day to drive their business.  I also learned that the data break down in Poland is 90% block, 10% file which I found interesting and would be interested to check back 12 months from today to see how it will be different.

I did learn something very interesting in Poland.  The question was asked “Why XIV”?  What is so special about XIV.  The answer was awesome.  The answer started with 2 questions:

1)      How old is RAID?

2)      How old is your iPhone?

The reality is data growth is out pacing what traditional RAID can handle and data profiles are changing as well.  These combined have driven new technologies like Cleversafe, Cloud Computing, Hadoop and XIV.  Just like the iPhone is a new approach to the smart phone based on new things we know about how these smart phones are being used, we know more about how data and storage is being used.  New ways to deliver capacity and performance are needed in order to keep up with the changing times.  I thought it was a very good answer in terms that make customers think.

Thursday evening I traveled back to Warsaw where I got in a bit late and just went to a local pub, Sketch.  Grabbed a small bite and some local mead and then headed back to the hotel.  I did get to see the local Palace of Culture and Science in the middle of Warsaw, very impressive, built as a gift from Russia to Poland.

I have an early flight to Prague.  I am very excited about this part of the journey as I have always wanted to travel to Prague.  Press meeting right when I land.  Stay tuned.

Today I begin a 12 day trip to Easter Europe to talk about IBM Storage.

The trip will take me to:

• Moscow, Russia
• Warsaw, Poland
• Prague, Czech Republic
• Ljubljana, Slovenia
• Umag, Croatia

In Russia, on September 6, I will be at the Information Infrastructure Conference and the following day meeting with customers to discuss storage and storage efficiency.

In Poland on September 8, I will be presenting IBM’s Real-time Compression at Storage University.

In Prague I will be meeting with the press as well as speaking with customers.  Additionally, I will be spending the weekend in Prague, a city I have always wanted to visit.

In Slovenia on September 14, I will be presenting at IBM’s Innovation Center at an IBM Solutions Event.

Finally in Croatia on September 15, I will be at the IBM Forum, the largest IBM even in Croatia.

In each location, I will be speaking with partners and customer on IBM’s innovation in storage, storage efficiency and Real-time Compression.  I am looking forward to learning what the largest storage challenges are across Eastern Europe and users go about solving their challenges.  Additionally, I will be doing some local enablement for our partners and sellers.

I will blog from each location.  I will talk about the professional part of my travels as well as, hopefully, one personal event.  I have tried to make sure that in each city I have time to do one interesting thing.  I don’t know when, if ever, I’ll be back to these cities and these are some places I have always hoped to go.  Too often we travel and its all business.

Also stay tuned, when I land I will have an update from my trip to VMworld.  It was fantastic.  Truly the best end user show around.  I learned a great deal and can’t wait to share some of what I saw.  As always – comments are always welcome.

This weekend I had an interesting conversation with my Dad.  We were discussing backup.  My dad basically runs IT for the State of Maine.  The State of Maine uses CommVault backup software.  So I posed the question to him, “What would it take for you to rip out CommVault and replace it with another solution.  He thought about it for a moment and replied “I wouldn’t”.  His answer came down to a couple of reasons.

First was the expense.  It’s not just about buying the new software, it would be training people to run the new software and it would be about throwing away the massive investment they have in their existing product as well as converting all the years of backup takes created with one software to the new software.  This is one of the biggest things vendors forget when trying to sell a customer on their backup software.

Second was the fact that, feature for feature, the top 5 traditional backup software products are not really that different from one another.  Sure, I do agree that some products have features that others don’t, and others products have features that work better than others, but in reality, the delta is so small and the workarounds are so simple it doesn’t really matter.  Unless your replacing traditional backup software with an evolutionary source based data deduplication software (which is only applicable for some environments) there is no advantage to switching software.

The challenge is if Data Protection is still one of the biggest and most expensive pain points within IT, how do the problems get resolved if replacing the software controlling it all is too costly to change?

It’s time to start asking new questions about how to solve the problem.  In the backup ecosystem there are a few things IT has to juggle in order to maintain equilibrium.  First there is data growth.  As data continues to grow, it gets more challenging to complete backups in the time available.  Second is availability, this is where the terms RPO and RTO (recovery time, recovery point objectives) come into play.  How quickly can I get my data back, and how old is it (how much data have I lost) when I do get it back.  Third is corporate governance.  Keeping the business out of jail, or from losing a law suit, is a key role of data protection and when done properly can actually help with backup.

So when trying to balance all of this, where does one start?  It starts with one key concept, What is the value of the data to the business at any given point in time? Based on this, IT can begin to develop a very effective data protection strategy while keeping the cost of doing so in check.

There are two things that must happen in the development of this strategy.  First IT needs to answer the data value question.  This can ONLY be done by having conversations with lines of business managers to understand a few key metrics.  Metrics such as:

1)      How much does it cost the business to be down?

2)      How much does it cost when data is lost?

3)      Does the company need to adhere to any regulatory requirements?

There are many other questions, that need to be answered but these are a few of what IT needs to be thinking about.  Next, IT needs to establish some goals.  These goals need to be realistic given the budget and they need to align to the value of the data.  In other words, apply the right technology to the right data based on its value to achieve balance.  These goals should take into consideration:

1)      How do I ensure the fastest recoverability reasonable for a given data set?

2)      How do I move as little data as possible (moving data is expensive)?

3)      How do I recover as little data as necessary (smaller amounts of data being recovered = faster recoveries)?

4)      How do I ensure I store data on the lowest cost medium practical to meet my SLAs?

5)      How do I delete data at the right time?

These are very lofty goals.  Keep in mind, unless your company is in the business of providing data protection services, your company doesn’t make any money backing data up.  Backup is an insurance policy.  The trick is to balance the costs of protecting that information with the value of the information.

Where does IT start?  Start with the goals.  If RPOs/RTOs are the challenge, try leveraging space efficient snapshots to have data local and more point in time copies for faster, “newer” recoverable data.  If backups need to be faster to help close the backup window, backup to disk, then copy that data to tape.  If that disk is too expensive, deduplicate the data.  To move as little data as possible and recover as little data as possible, leverage real-time compression on your primary storage.  By compressing data as much as 5x, AND ensuring that the compression solution works with existing data protection processes, there will be a significant reduction in the amount of data that needs to be moved and stored in the backups giving back a great deal of time in the backup window.  Next, think about archiving technologies.  Most data retention practices have a schema similar to 14 dailies, 4 weeklies, 11 monthlies and depending on the yearly retention practice X number of yearly’s.   A proper archiving technology can take the multiple copies of data being stored in the “yearly” data set and reduce it significantly.  By taking that data out of the backup flow, again, the costs of disk and tape storage go down and moving less data gives back time in the backup window.  Also, setting the proper archiving policies will identify when to delete data which saves on storage costs, both the medium and the footprint (tapes at an offsite facility).

At the end of the day, there are a number of solutions available to customers that can have a profound impact on your business without having to rip out your existing backup software that don’t cost a lot of money and won’t impact the existing investment.  I encourage data protection professionals to work with vendors who take a more consultative approach in helping to solve the data protection problem.  Work with a vendor that has a very large install base who has implemented a number of data protection solutions and can draw on a wealth of use cases and best practices collected over time.  Solving the data protection challenge is not easy, to solve it properly, you need to properly align the value of your data to the technology used to protect it and most of all it takes a good deal of experience reach out to the pros.

In May I delivered a presentation to a number of EMC customers on the topic of Data Deduplication 2.0 - Comprehensive Capacity Optimization.  The point of my presentation was simple (and keep in mind this was before the Data Domain acquisition); there are a number of capacity optimization technologies/capabilities that are available to customers today.  Originally these deduplication technologies were used primarily for backup purposes but slowly, deduplication is making its way into primary storage. Deduplication in primary storage makes a lot of sense FOR DATA THAT IS STATIC.  Why only static data?  Static data is data that isn't used frequently (doesn't mean it's not important, it just simply is not accessed often); because access to this data is infrequent, the performance requirements for this data is less than that of active data. Remember; nothing in IT is free.  If I deduplicate data, in order to use it, I must ‘rehydrate’ it and thus there is a performance implication so I want to be careful where I deduplicate data so as not to inhibit performance on production data.

Dr. Dedupe and Tom allude to Deduplication 2.0 moving beyond backup storage and into primary storage.  While deduplication in primary storage is technically possible, it is important that customers understand two important points:

1) Performance: whatever I do to deduplicate (I like optimize) capacity in order to save space, I must ‘undo’ in order to use the data.  If I set a policy that says any data that is 30 days old can be ‘optimized’, I need to be sure that data 30 days old is not active or I could pay a substantial performance penalty when using this data.  I may set a policy ‘any data that hasn’t be touched in 30 days, can be optimized.  I would just want to make sure that there is no scenario where at the end of a quarter let’s say, I would need to rehydrate all data in order to run some report.

2) Comprehensive and cumulative deduplication throughout my storage tiers.  What do I mean?  If I compress and single instance (deduplicate) data on my primary storage utilizing one set of deduplication technologies, say single instancing and compression algorithms, and then I backup this data using sub-file deduplication, a separate set of algorithms, then what I am left with are two separate sets of deduplicated data silos, and no one wins in this scenario.

It is important, no matter what deduplication technology you decide to use, that you can actually leverage the data stored in the deduplication device and that as data moves from device to device it doesn’t need to be rehydrated before it is moved.

A great use case of capacity optimization in primary storage is how EMC evolved the Celerra product this year.  Through a policy, let's say any data that is older than 30 days, is compressed and stored as a single instance, with users seeing as much as 30% to 50% storage savings.

The real goal of Deduplication 2.0, and I think Dr. Dedupe alluded to this in his post "The Dedupe 2.0 Pundits Are Still Swimming in Lake 1.0" is that customers win when deduplication technology is a part of the core system or file system, when I no longer need to rehydrate data as I move it from primary storage to secondary storage.  If each storage device in the 'stack' understands the language of the device in the stack ahead of it and the 'deduplication' or file system is coordinated and cumulative from device to device than the customer is the winner.  This pertains to primary storage, backup storage and archive storage.  Never having to rehydrate data allows for more efficiency and a reduced tax on devices that can save the end user money.

Tom Cook, CEO of Permabit points out in his blog post "Dedupe 1.0 vs. Dedupe 2.0: The debate ensues" that the only value to deduplication for primary storage is to move your data to a deduplicated archive which allows you to store data, efficiently, long term which I agree with, but as we have seen, not that practical.  Why? Because at the end of the day, the costs to manage storage are going up, up, up and the costs to buy storage are going down, down, down.  End users (NOT IT) are generally lazy or should I really say, just too busy to manage this storage.  In order to properly archive data, you need to have a policy that tells you what to move and when to move it.  IT can make all the recommendations in the world about the value of archive, but if users or really, lines of business managers don't tell IT what data is important and what can be archived, then IT doesn't really have a choice, which makes the premise of moving data to an archive, deduplicated or not – moot.

The real issue is balancing capacity optimization (to what granularity you deduplicate data) against performance on the appropriate tier of data, given that deduplication will happen on all tiers of storage.  The higher the performance requirements (tier 1) the less 'optimized' I make the data, the lower the performance requirements (tier x, archive) the more optimized I make the data.  The benefits to the customer are that I can A) optimize data, consistently among each of its devices, and B) it can be cumulative from device to device, removing silos of deduplicated data across the stack.

For more on tiered dedupe, read my Betamax Redux blog post on EMC's vision for deduplication and hopefully this will put you on a high performance ‘Road to Recovery’.

]]> http://www.thestoragealchemist.com/comprehensive-capacity-optimization-deduplication-2-0/feed/ 1 http://www.thestoragealchemist.com/a-data-protection-reference-architecture-the-final-chapter/ http://www.thestoragealchemist.com/a-data-protection-reference-architecture-the-final-chapter/#comments Tue, 01 Sep 2009 13:01:10 +0000 Steve Kenniston http://www.backupandbeyond.com/?p=359 The Architecture

This ‘architecture’ diagram, as you can see, is not a typical architecture diagram, but hopefully it can be used to align your business and business objectives with the technologies that are available and can best be applied to solve your issues helping to balance, cost, complexity and compliance.

This diagram can also be used to do a couple of other things.  It can help you begin to classify your data and align your  data to your business objectives.  It also lets you begin to identify what data or data services in your environment that may be more important to you than others and based on this help you to choose areas you may want to outsource or move to the cloud.

As you can tell, there really is not one solution for meeting all your data protection needs.  The challenge comes with managing multiple solutions in an effort to meet your business objectives.  While there are only a few technologies available that allow you to manage your environment across all your RPOs and RTOs, it is important that I point out EMC’s NetWorker is able to do this, centralizing your data protection infrastructure  for ease of management.  It allows you to manage traditional backup, source based deduplicated backup with Avamar, CDP with RecoverPoint, as well as the EMC disk libraries and tape where the data is stored.  Now, I am not saying that NetWorker solves all of your data protection challenges, nor am I suggesting that replacing one traditional backup technology for another is the right answer, but what I am saying is that if you’re looking to have all the feature functionality required to meet all your business objectives and you want easier management, NetWorker is one avenue to get you there.  Additionally, the underlying image of the triangle represents data protection management.  Putting all the new technology in place is one thing, managing it, and ensuring you are now meeting your business needs is another.  EMC's Data Protection Advisor can help here as well.

This diagram can help customers layout a new, better data protection schema for their environment and start thinking about data protection a bit more strategically versus tactically.  It can also help vendors speak to customers about how they should look at their environment in order to identify specific challenges and the means they need to alleviate these challenges , taking backup, beyond.

The tip of the triangle focuses on the applications (or data) that drives your business.  It is these applications within your business that, should they go down for any length of time, cost you money.  The recovery of this information, in the event of a ‘disaster’, needs to be very fast (RTO in minutes) and the data can’t be very ‘old’ when it is recovered (short RPO, less than 24 hours).   Typically,  the technologies that are used for these types of applications are replication (synchronous or asynchronous) or continuous data protection (CDP).  These technologies ensure that recovery at the alternate location  are instant (or near instant) and / or give users the ability to pick a point in time they want to recover to in order to ensure no data loss and the ability to bring up the applications as fast and accurately as possible.  This category, much like the rest of them, have the same disclaimer, 'one size (product) does not fit all'.  Depending upon the value of the data in this tier, and the risk to the business if this data is unavailable drives the technology and spend in this part of the triangle.  Keep in mind, the right technology (Don't choose CDP if you need an active remote file system) gives you the best recovery (RPO) for your business needs and can keep you on the Road to Recovery.

Let's take a look at a simple backup environment.  We won't worry about archiving or compliance for the moment, just operational backup and recovery.  Today's backup has a number of complexities.  There are some data sets that have weekly full backups and daily incremental backups.  There are some data sets that sit under applications that, for faster recovery capabilities and simplicity, require daily full backups.  Once the backups are done, in order to ensure true data protection reliability, a process of checking the backup logs to ensure every system was successfully protected begins.  Next, backup tapes are either created (if it is a disk based backup) or tapes are taken from the library and moved to a transportable box, hopefully a secure box.  Finally, a third party vendor comes to pick up the tapes and take them off site for safe-keeping.  Additionally, if the data is backed up using encryption, then the encryption keys are also kept off site for security purposes.

 Customers face these standard backup challenges:

1) Backups take too long and cannot meet backup windows as a result of too much data.

2) Backups fail due to poorly configured (networked) backup environments.

3) Backups at remote offices are 'unreliable'. (Don't follow best practices set in the data center.)

a. No one with the appropriate skill set is available to monitor these backups.

b. No one with the appropriate skill set is available to troubleshoot these backups.

c. No one with the appropriate skill set is available to perform data recovery.

4) New applications / processes cause additional challenges; does this application need incremental backups, full backups, what is the RPO / RTO???

5) Managing backup tapes is too difficult and costly.

However, the reality is that in this particular IT shop, no one has ever been fired for data loss. Each time there is a recovery request, data is recovered.  It may not be the absolute most recent data, or it may take 48 hours to recover, but eventually, the data is recovered. The question is, has everyone's business objectives been met? Chances are the answer is "no" but when the issue of what it would cost to meet everyones' needs comes up, there is usually no money in the budget for 'backup' and it's right back to the same old way of doing things. Backup is not really strategic to a business (unless of course you're in the business of providing backup solutions to customers) but it is more of an insurance policy. There is no doubt you need it, but you want it for the lowest possible price, hope you never have to call on it, and when you do, you better get good service.

Maybe that is why EMC is now the GEICO of data protection.

 That aside, when there is money in the budget, it usually comes in small doses so backup administrators have to make the biggest impact in the 'easiest' way possible. This means, implement something that allows them to meet most of their challenges and doesn't:

1) Change process because they already have run books established for data recovery and because everyone is already trained on the existing technology.

2) Change configuration because they have already invested a great deal of time and money to sort out their issues with the existing products.

3) Cost a lot of money

That usually means, augmenting the existing backup software technology with something that allows them to gain some efficiencies on the backend because they already have significant investments in their backup software. This was one of the main reasons for the success of VTL (virtual tape libraries). It is way easier to unplug the slow, serial tape library and replace it with fast, parallel disk. The backup administrator gets all the advantages of disk and doesn't have to change a single process, except for maybe adding a step of cloning the data from the disk that looks exactly like tape, to an actual tape in order to offsite the data. Additionally, this is why companies with target deduplication devices became so popular so quickly. When VTL was having challenges solving backup data capacity issues, deduplication became the next popular thing.  The big issue was plugging into the existing infrastructure without disruption.  If I have to change too much about my process, I can't 'afford' to make it work.

The trouble is backup administrators are at an inflection point. They can no longer continue to use the same old technology at the front of the backup process and meet the needs of the business. We are at a time when new technologies such as source based deduplication technologies can really have a significant impact on a number of the backup challenges. The problem is that it goes against the grain of why IT doesn't want to change technology, because it forces a change to the process. For example, out come the traditional backup agents and new ones are put into place. Since data no longer is stored in tape format, new processes must be utilized for getting tape offsite. When backup administrators hear this, they tend to shy away from it. It costs money and it changes processes right when they had all the original processes figure out.  It is only now that source based deduplication solutions have gained significant momentum as it is really solving a number of the key data protection challenges for more than 70% of the data in most data centers.

• Remote offices can now experience the same set of data protection best practices that are used in the data center. (Keeping in mind, IT is accountable for 100% of the data created in the corporate, local or remote.  This is good piece of mind.)VMware environments tend to ruin a TCO when using traditional backup applications. Leveraging source based deduplication can bring up your TCO and ROI.

This is not to say that source based deduplication is the savior of the backup world. It is not. There are places where source based deduplication technologies are not the best fit. Very large environments with very high change rates and little duplicate data don't tend to be good fits. However, if you attack the places that are a good fit for source based deduplication, you will create relief in your backup environment at the target and that will be good for everyone.  It is time to take backup, beyond.

Posted by Steve Kenniston