Today I begin a 12 day trip to Easter Europe to talk about IBM Storage.

The trip will take me to:

• Moscow, Russia
• Warsaw, Poland
• Prague, Czech Republic
• Ljubljana, Slovenia
• Umag, Croatia

In Russia, on September 6, I will be at the Information Infrastructure Conference and the following day meeting with customers to discuss storage and storage efficiency.

In Poland on September 8, I will be presenting IBM’s Real-time Compression at Storage University.

In Prague I will be meeting with the press as well as speaking with customers.  Additionally, I will be spending the weekend in Prague, a city I have always wanted to visit.

In Slovenia on September 14, I will be presenting at IBM’s Innovation Center at an IBM Solutions Event.

Finally in Croatia on September 15, I will be at the IBM Forum, the largest IBM even in Croatia.

In each location, I will be speaking with partners and customer on IBM’s innovation in storage, storage efficiency and Real-time Compression.  I am looking forward to learning what the largest storage challenges are across Eastern Europe and users go about solving their challenges.  Additionally, I will be doing some local enablement for our partners and sellers.

I will blog from each location.  I will talk about the professional part of my travels as well as, hopefully, one personal event.  I have tried to make sure that in each city I have time to do one interesting thing.  I don’t know when, if ever, I’ll be back to these cities and these are some places I have always hoped to go.  Too often we travel and its all business.

Also stay tuned, when I land I will have an update from my trip to VMworld.  It was fantastic.  Truly the best end user show around.  I learned a great deal and can’t wait to share some of what I saw.  As always – comments are always welcome.

IBM @ VMworld 2011

After a full first day at VMworld, I started to think more about IBM and their technology solutions that help customers in a VMware environment.  Here is a top ten list of things to consider when looking at a VMware implementation and how IBM can help.

#1 Integration

VMware is playing Switzerland and ensuring all vendors are on a level playing field, so when other vendors state that they have “better” or “closer” technology integration than other vendors its probably not true.  Some vendors may not choose to integrate with certain things, but rest assured, all of   VMware’s APIs are open to all vendors.  Take a look and see how IBM is providing plug-ins for vSphere, SRM, and VAAI in XIV as well as other storage platforms.

#2 Ease of Use

IBM has seen, firsthand, a number of our customers switch from different competitive platforms to XIV because of the simplicity of the XIV solution.  A large manufacturer is one example of a customer who is provisioning new VMware instances in less than five minutes with XIV. 

Another XIV customer, who is a very experienced storage administrator, saw the XIV GUI and quoted "I don't get it (XIV GUI).  It can't be that easy.  Either I'm missing something or they are not showing me everything."  Well, the reality is, it is that easy and that interface is prolific throughout the IBM storage portfolio including the Storwize V7000 and SVC.

#3 Storage Efficiency

Probably one of the most important topics this year is Storage Efficiency and IBM is a leader in this department.  The N-Series with the Real-time Compression appliance can reduce the VMware storage footprint up to 75%.  Users tell us that by implementing VMware, their storage footprint has grown by as much as 4x.  Therefore their overall IT budgets didn’t get better, the dollars just shifted from servers to storage.  IBM’s Real-time Compression users can save up to 75% without any performance impact.  Additionally, Real-time Compression is the only compression technology that works in conjunction with deduplication, compressing the data before it is dedplicated, giving an added benefit to the technology.

Now users have the opportunity to get their overall IT budget back under control.

#4 Data Protection

The reality here is that many enterprises are waiting for the war to be fought out between the vendors in this space, or looking to embedded snapshots and disk based technologies with mirroring to cut out all of the host based challenges with data protection. 

A report by Taneja Group, sponsored by multiple clients, suggests that the biggest issue in virtual environment is data protection as many enterprise do not know what they need to do and they are looking at their current vendors to provide solutions.  So work closely with the IBM team and leverage all of the work that IBM has done with Tivoli and VMware to help solve your data protection challenges. 

A lot of folks like to talk about deduplication when it comes to VMware, just make sure it is implemented properly and at the right place.  ProtecTIER has great deduplication ratios and great  performance. 

#5 Flexibility

I am not sure how you get more flexible then working with IBM.  From hardware to software to services to partners, IBM offers solutions across a wide spectrum.  Whether it be hardware solutions that can meet a range of performance requirements and application types, to software that can help users analyze their data more effectively.  IBM can also deliver all of these solutions through our relationships with or ISVs as well as partners offering superior flexibility. 

#6 Availability

When it comes to high availability in storage, it is hard to beat the new V7000 or the XIV product.  Innovatively designed specifically around high availability, users can move to a virtualized storage platform such as XIV and users can see the real-world of availability and reliability that does not sacrifice performance in any of their applications. 

#7 Scalability

With IBM XIV, you can simple scale as you need to and automatically and take advantage of new capacity and linear performance improvements as well as managing the entire enterprise from a single, easy to use GUI.

Also, with Real-time Compression, you now have the added benefit of putting more capacity in your existing footprint to do even more analytics while saving on footprint, power and cooling – all in real-time.

#8 Services / Solutions

IBM is the worldwide leader in providing services.  IBM is the largest OEM of VMware solutions on the planet and provides support and services in 170 countries around the globe.   IBM’s Global Services team has architected and installed hundreds, if not thousands of VMware implementations, helping customers go from a non-virtualized to a virtualized world.  IBM, as well as its partners, can help migrate customers to a virtualized environment without a long outage and maintain application and customer production.

#9 TCO / ROI

IBM offers great solutions that reduce the risk, cost, and complexity of the virtualized world.  IBM focuses on the real-world customer challenges.  Customers have been hit hard these last few years when it comes to budgets in order to manage their IT environments.  We keep helping our customers do more with less by enabling a more efficient storage platform than any other vendor.  IBM XIV, V7000, N-Series, SVC and ProtecTIER solutions are great fit for solving difficult VMware challenges and we have real-world references to prove it.

#10 100 Years of Innovation

The bottom line: there is always more to do, IT changes at a rapid pace and it is the vendors job to keep up with the needs of its customers.  IBM has been doing this for 100 years and we will continue to do so.  

For more on IBM and Storage Efficiency at VMworld, see us on SiliconAngle.tv live this week at 3:30 PT.

History truly does repeat itself.  We are talking about the history of data storage.  Every once and a while a new technology comes along that requires a new way to think about infrastructure.  Notice I said “infrastructure”.  I’d like to paint two analogies:

Analogy 1: RAID – Prior to RAID users stored their data on disk and if they could afford it, they backed that data up to have a protected copy of their data.  When RAID came out, users were able to store their data on multiple disks appearing as one device.  The benefits to this were, increased data reliability, better performance.  This new technology however, fundamentally changed how disk was sold, but the questions were the same:

1. How much capacity do you need?
2. What type of performance does your application require?

The sales reps point of view changed.  There were a number of new considerations that needed to be taken into account.  First, the age old question, “Will I sell less storage “stuff?”  Remember the person, at the time, selling the disk was probably also selling the backup tape and software to protect that information.  If the disks are more reliable, maybe the customer won’t need as much tape?  Second, when the capacity question came up, the seller also needed to know what type of RAID the customer wanted to ensure they sold them enough drives.  It was no longer as simple as asking the capacity requirements and dividing it by the drive capacity at the time.  Now depending upon RAID levels there was a new set of math that needed to be done.  Third was the notion of performance and more spindles meant more performance so now that the capacity equation was solved for, you also needed to know the I/O requirements in order to make sure the right number of drives were sold to solve for the capacity as well as the performance.

Guess what, we figured it out and the industry never looked back.  RAID is a defacto standard in all storage subsystems today, I even run RAID in my home.  The business benefits of having RAID far outweighed the costs.  In fact, it is probably one of the first times in storage history that the question of, “how can you afford not to have it”, came up.

Analogy 2: Virtual Machines – When VMware came out the value proposition was, do more work, with less physical infrastructure.  And again, the business benefits far outweighed the technology hurdle of implementing the new solution.

Keeping in mind that it is much harder to change process in IT than it is to change technology, IT decided that this new way of serving up processing power to applications was well worth all of the process changes that it would require.  One example, backup would need to change when implementing virtual server technology.  The data would grow 4x and the processing of that information for backup would take longer, in a world where time was all to valuable.  However the business benefit justified the change.

Again, the sellers questions were consistent:

1. How many virtual servers do you need? (Capacity)
2. What type of performance do you need for each virtual server?

The answers to these questions allowed a sales rep to configure the right number of physical systems to handle the right number of systems to make the line of business successful.  Additionally, some of the same considerations came up.  “Will I sell less server and make less money?”  Now that there was new server technology (more processors, the ability to handle more memory) systems could be bigger, and more expensive.  Sellers also needed to know a bit more about “capacity”, how many virtual systems could a physical system run successfully?  They also needed to have an understanding of performance.  Now sellers were configuring systems to run the equivalent of 20 to 100 servers on one system.

Today I would suggest that we are at a cross roads in history. New technology has come along that will have a significant impact on the storage world.  First, research from IBM reflects the fact that disk drives can no longer keep getting two times as dense for half the cost as they had been throughout the late 90’s and early 2000’s.  The technology doesn’t exist today to make the drives spin faster, stay cool and not loose data.  Until now.  Real-time compression is a game changing technology that will add significant value to the storage industry without having to change the way IT thinks about the deployment of their storage.

Data is growing at such a significant pace today and with the latest IBM research about disk capacities, something needs to change.  Data centers are just running out of space and more customers want to keep more data on line for reasons such as competitive edge or compliance, but no matter the reason, they want access to their information.  Enter real-time compression.  Now there is a fundamental difference between real-time compression and other compression technologies and compression implementations but I am not going get into it here, but it is safe to say that post process and in-line compression are very different than real-time compression and users can’t get the benefits of improved primary storage capacity, transparently, with no performance impact with anything but real-time compression technology.

Again, real-time compression, like other game changing technology, doesn’t require any new questions; there are just simply a new set of math equations.

1. How much capacity is required?
2. What is the performance requirement?

In time, real-time compression will be as ubiquitous as RAID, and just like users don’t think that much about RAID, users won’t need to think about compression.  Compression will become an expected feature of the array.  It doesn’t matter that it now takes fewer drives to satisfy the original question around capacity and performance.  With data growing as fast as it is and with disks not being able to keep up their growth pace, something needs to change and that something is real-time compression.  Soon, it won’t matter what the physical disk capacity is of a disk drive, it will be about a disks virtual disk capacity, what it has the capability of storing that matters.  It is time we all started thinking this way.

“Storage Efficiency” has become a big topic over the past 12 months.  There are a number of new technologies that have come out in the last few years that are helping to deal with storage growth.  We all know that data is the root of the decisions that drive business today.  The more data you have, hopefully, the better decisions you can make to drive your business to success.  The question is, “what is the value (and hence the cost) of the infrastructure to create that success?”  What we do know is that the ability to put more data in a highly efficient footprint can give your company a competitive edge.  There are five technologies that can help an IT organization create an efficient storage infrastructure.  These are:

1)      Tiering

2)      Virtualization

3)      Thin Provisioning

4)      Compression

5)      Deduplication

It is also important to point out that there are some semantics when talking about storage efficiency, specifically between efficiency and optimization technologies.  I think it is useful to attempt to define these as they lead us to picking the right solutions for what we are trying to accomplish.  For the purpose of this post, efficiency will relate to making existing capacity more useful and optimization will mean making more capacity out of existing capacity.

Using these definitions, technologies such as Tiering, Virtualization and Thin Provisioning are efficiency technologies.  These technologies help to utilize the existing capacity that you have.

Tiering is technology that is used on about 10% of your data or less.  It is used to move data that requires higher performance to flash storage.  Good tiering technology analyzes data access patterns and moves the most active data to the highest performing disk.  It doesn’t really change the amount of physical capacity that is required; it just changes what type of capacity is required and allows IT to make sure data is operating as fast and efficiently as possible.

Virtualization technology allows IT to make sure disk utilization is used as efficiently as possible.   Until recently storage utilization rates were around 50%.  By leveraging virtualization technology, IT can group pools of storage so they don’t need to purchase capacity needlessly.  Virtualization can be used on 50% to 60% of your storage but it doesn’t change your physical capacity infrastructure requirements and at most allows users to take advantage of 20% to 40% of their capacity that they once didn’t access.

Similar to Virtualization technology, thin provisioning technology also can be used on 50% to 60% of your capacity however, thin provisioning technology gives IT about 10% to 40% of their capacity back.  Thin Provisioning helps IT manage their existing capacity and their utilization by being able to make capacity available to users much easier again however it doesn’t change the amount of physical storage infrastructure required.

Optimization technologies help IT to better manage their physical storage footprint.  Optimization technologies optimize existing infrastructure by allowing users to put more capacity in the physical same space.  The two technologies that are currently used today are data deduplication and real-time compression.

Optimization technologies are a bit tricky.  There is a balance that is required between optimization and performance and availability.  At the end of the day, IT chooses the storage it buys with two very important characteristics in mind, performance and availability.  Optimization technologies can not affect these characteristics.  It is for this reason that data deduplication really isn’t ready for “prime time” on primary, active storage.  Data deduplication creates too much of a performance impact on primary, active data.  Today, data deduplication could be used on about 10% to 15% of the primary, less active capacity that is in the data center and only provides about 30% to 50% overall optimization.  In other words deduplication technology can impact the physical infrastructure by as much as 10%, meaning IT may not need to buy as much physical capacity.

Real-time compression, on the other hand, has one of the most dramatic affects on primary storage capacity.  Real-time compression can be used on as much as 85% of the storage footprint and can compress data between 50% and 80%.  That said Real-time compression could have IT purchase as much as 70% less overall storage capacity.  Real-time compression also does not affect the main characteristics for which users buy storage (performance and availability).  IT could have as much as 70% less footprint but keep the same amount of data or more on-line.  Additionally, IT can now purchase storage opportunistically without having to have such a dramatic impact on their infrastructure, process or budgets.  This allows companies to keep more capacity on line and available to help companies do more analytics on more capacity and become more competitive.

When deciding which storage efficiency technology will have a more effective impact on your overall environment and budget, start with optimization technologies and start to get the data growth under control.  Adding value to the line of business that can drive revenue with more data will make you a hero and your business more successful.

See how data deduplication and IBM Real-time Compression work hand in hand.

]]> http://www.thestoragealchemist.com/storage-alchemist-video-update-2/feed/ 0 http://www.thestoragealchemist.com/top-10-reasons-real-time-compression-provides-extraordinary-storage-efficiency/ http://www.thestoragealchemist.com/top-10-reasons-real-time-compression-provides-extraordinary-storage-efficiency/#comments Wed, 15 Dec 2010 17:14:44 +0000 Steve Kenniston http://www.thestoragealchemist.com/?p=1011

Over the past few weeks I have witnessed the proverbial mudslinging that takes place in the blogosphere when marketing feathers are ruffled.  Most recently I was reading Rich Anderson of The StorageSavvy Blog.  The article was "Compression better than Dedup?  NetApp Confirms!"

I have to agree with Rich on many fronts.  First, "When all you have is a hammer, everything is a nail."  Rich points out vendors have to sell "what's in the bag" so it is conceivable that all problems look like they can be solved with their solution.  If you look back over the last few years NTAP has always had a "me too" reputation.  Whatever the industry has, they have one too and its better.  For the last few years, while competing against Storwize, they have pulled the EMC tactic of trying to stall a market by saying, "We have optimization for primary storage with deduplication."   The reality is, you can't use it in real time, it is a resource hog, and again Rich mentions, the only use case it works well on in primary storage is VMware (and that is ONLY IF the customer stores their data outside the .vmdk file otherwise compression is much better).  Now that NTAP has compression their story has changed saying that compression on primary storage is better for most use cases.  Duh!  The folks at Storwize (now IBM Real-time Compression) have been saying that for years.  Why, deduplication is great for repetitive data sets, i.e. backup, not primary storage.  There just isn't that much repetitive data in primary storage.  Again, NTAP is trying to stall the market saying they have "in-line" compression for primary storage.  Sorry guys, not good enough.  In-line is NOT Real-time.  Rich also points out that the key characteristics of storage for customers are capacity and performance.  Patrick Rogers of NTAP has said publically that compression WILL indeed impact performance and that they even have a tool that will tell you how much performance will be impacted.  While NTAP may say compression is "free", we all know nothing worth having in life is free, you get what you pay for.  If you need the performance to do compression you are going to have to perform a major upgrade to  your filer in order to just be able to perform compression let alone try to do compression in real time.  No real savings there.

It’s a simple fact that embedding optimization technology into the array you are optimizing is the right direction, it is just not ready for prime time today for a number of key reasons.  If you can't implement the technology in the array and preserve performance, availability, price and feature set, then its not a viable solution.  I know that every answer in IT is "It depends", and I am sure there are some use cases where the solution is "good enough" for some customers who meet all the proper criteria, the reality is, it is not ready for the general public.

On the other hand, if you can off load the work from the array in an appliance model, preserve performance and ensure 100% transparency to applications, infrastructure and processes then that is a real solution.   This is exactly what the real-time compression technology does from IBM.  Yes it is an appliance that sits in your CIFS/NFS storage infrastructure.  However, the benefits of the time based compression technology, along with the ability, by offloading the array from doing the compression, to maintain and in some cases increase performance, and the fact that putting the technology in place requires no change to your applications, no changes to your storage infrastructure and no changes to your downstream processes means this solution is the top primary storage optimization solution on the market.  It's not a big surprise.  First the technology comes from an Israeli startup, some of the smartest minds in storage / high tech.  Second, most great solutions start out as an appliance.  They are easy to deploy and allow you to get an understanding of how the technology really works in multiple use cases before trying to embed the solution with limited information.

Below are the top 10 reasons to deploy a Real-time Compression appliance for NAS from IBM over NTAP compression technology:

Of the customers running the technology in their environment over 57% of these customers have rated the Real-time Compression technology an "Excellent value – one of the top values in my data center" or better.

I am guessing by David Vaughn's visceral reaction that Rich hit a nerve that is more than just the typical old NTAP / EMC battling.  I don't think anyone ever told David that usually when you have such a reaction to something, you must be trying to hide something.  In this case, while NTAP has great products, they fall short when it comes to optimization technology.  Also, would you want your migration process to look like this?

Between time off with the family this summer and all the work required to get done between 'signing' a deal to be acquired and 'closing' a deal to get acquired, the blog has been a bit slow.  But I am here now to tell you it is official.  Storwize is now Storwize, an IBM company.

As for myself, I am looking forward to the work of integrating the Storwize Technology into the IBM Storage portfolio.  The Storwize group will live under the STG organization under Brian Truskowski.  There is a new ground swell taking head at IBM these days all around storage efficiency.  To get a better understanding, please have a look at my new colleague, Tony Pearson's blog discussing storage efficiency.  My job will be now to evangelize how IT now needs to take a look at all of the available storage "services" (clones, snapshots, thin provisioning, replication, compression, deduplication, etc...) can help to create an overall storage solution that allows them to reduce their over all $/TB on not only capital expense, but also on operational expense.

Lets face it, data growth isn't slowing down and there is never a one size fits all solution for storage.  The great part about being a part of IBM now is that we have all the tools to pick from to architect a data storage solution, end to end, that allows customers to reduce their overall $/TB for both primary as well as secondary storage and make that storage much more efficient and work for the end user.

This is going to be an exciting time.  I am also anxious to continue the Storage Alchemist blog.  EMC, under the guise of Polly Pearson and Chuck Hollis taught me that social media is great, but social media done right, in a collaborative and thoughtful way can drive influence.  I join some of the best bloggers around from IBM.  (I have added Tony's "Inside System Storage" - It is a great read.)

The basics that we know about today with regard to the acquisition is that this is good for Storwize employees as they get to continue working on a great technology that they have matured over the last 5 years and will continue to mature.  This acquisition is great for IBM and Storwize customers as for IBM it adds the ONLY real-time compression technology to their storage portfolio driving storage efficiency.  And for Storwize customers, you now have the backing of a global 2000 company that is committed to continued development of an already great product as well as now have the resources to go after new features and functionality that often comes at the hands of limited budgets in a stratup.

For IBM resellers and partners, hold on to your socks because now you have one of the most competitive technologies in the market that will allow you to 1) provide a great service to your customer through efficient storage and not just sell customers more disk drives (I have talked to over 600 customers in the last 2 years, customers don't want more disk drives, they want smarter ones) 2) a competitive, value added solution that allows you to be much more competitive in selling your clients better solutions that aren't wrapped around infrastructure, but around business solutions that allow your customers to be more successful and happier and you to make more margins.

Seems like a win for all.

Over the course of the next few weeks we will be learning more as a collective team on what the roadmap looks like for IBM Storwize but we are all excited.  Stay tuned.

This blog will be taking a turn to more on storage technology can turned into IT Gold!

Rather than leave a lengthy comment on Tom Cook’s blog post from Friday Compression and Dedupe: Business Value and Data Safety (and from a marketing perspective, Friday’s are bad days to post blogs – especially in the summer) – I thought I would respond here (this may get lengthy as Tom made a number of points which I need comment on).

The first thing I do want to say is that when doing technical marketing; the proper strategy would be to not be on defense but rather take an offensive approach.  However, given the amount of FUD that Tom put in his latest blog post, I have to defend compression to some degree.

Now, I think we can all agree that data compression and data deduplication are two technologies that can complement one another very well.  Avamar (EMC) deduplicates the data at the source and then compresses the data before sending it to the Avamar Data Store gaining tremendous efficiency in network utilization.  ProtecTIER (IBM) compresses the data once it is deduplicated at the target device before it stores the data.  Other solutions also combine compression and data deduplication.

I’d like to comment on some key point Tom made in his piece where he is just blatantly wrong:

1)      Compression identifies redundant data across a very small window, usually 64 KB. – While this may be true for other compression technologies, this is not true for Storwize.  Storwize performs compression where the initial window is not fixed in size at all; it is the resultant write that is fixed in size.  This size is also specifically mapped to the I/O patter of the data being written.  The goal is such that in 1 I/O Storwize can do all the work it needs to on a particular file or LUN and it is for this reason Storwize has no performance penalty.

2)      Compression produces data reduction rates at most 2X for most data types. – Seems Tom needs a lesson in the most common answer in IT – “IT DEPENDS”.  Data compression ratios are 100% tied to the data type.  For a true indication of data compression ratios see Figure 1.

Figure 1

1)      Compression alters the underlying data structures and requires compression and decompression of data. – If you look up the definition of LZ Compression (in Wikipedia) you get the following: The Lempel-Ziv (LZ) compression methods are among the most popular algorithms for lossless storage. Deduplication is more of a lossy technology as it throws away pieces of data from other files in order to deduplicate data.  If any piece of data cannot be reconstructed for any reason, multiple data sets are now corrupt.  Additionally, in order for ANY application to work with any data it must be rehydrated, today’s applications must work with data in their native format.

2)      Compression operates in the data path and impacts read/write performance as a ‘bump in the wire’ (kudos to Storwize for their work to improve performance). – So we actually do thank Tom for his kind (and accurate words here).

3)      Compression is a potential single point of failure for data retrieval. - How would it be different for data deduplication?

Tom also provided a chart that had a few inaccuracies in it so I took the liberty of fixing some of these as well.  See Figure 2

Figure 2

I also want to call Tom out regarding his comment “I am gratified by the response to Albireo by … the press by recent OEM adoption.  Albireo is becoming a standard in data optimization…”

Here the term delusional comes to mind only because the links that Tom calls out are links that are focused on Permabit and not OEMs discussing the adoption of deduplication.  Additionally, when Permabit announced Albiero there were no OEMs mentioned or quoted as having adopted or even tested the solution.  This brings into question how the solution (which was only announced a week ago) is becoming a ‘standard’?

When it comes to providing technical information in any type of written communication, it is important to make sure, if you want to be credible, that the data provided is accurate in order to ensure credibility.  A lot of times this required doing research to ensure that the answers you are providing are correct.  It is clear that the competitive team or technical marketing team at Permabit has let Tom down here and not provided him with the correct information with regard to how compression works in real life (at least from a Storwize perspective).

This brings me to the tail end of the title of this post.  Perhaps a CEO should focus on running their company and driving the ‘OEM’ deals they speak of and doing business development than trying to do any type of marketing or technical marketing.  I know our CEO says that he is not a blogger and that some companies have natural bloggers while others do not.  I guess this is because he is too busy running the only company that delivers real-time data compression without any performance degradation and helping us to manage hundreds of customers as well as a number of OEM opportunities.

]]> http://www.thestoragealchemist.com/marketing-fud-and-doing-what-you-do-best/feed/ 10 http://www.thestoragealchemist.com/setting-the-record-straight-on-backup/ http://www.thestoragealchemist.com/setting-the-record-straight-on-backup/#comments Tue, 11 May 2010 16:04:38 +0000 Steve Kenniston http://www.thestoragealchemist.com/?p=830

Or should I say, ‘Setting the Record Straight on Backing Up Optimized Data’?  Carter discusses on this blog they myriad of ways to perform backups on optimized data.  (His blog actually reads more like a white paper explaining how backup needs to be configured to work with his product.)  One of the ways Carter describes to do backup is via NDMP and says “… is the most complicated.” The funny thing is that this is the way that 90% of enterprises backup their NAS data.  The other scenarios are not quite stated correctly or are again designed to lead users to believe their solution is ‘simple’ when they really add complexity (however, I’ll let the backup community debate that – I have been in backup for 10+ years and I know this won’t go over on them, nor do I want to waste too much blog space).  Finally the last scenario they discuss isn’t backup – its replication, but I’ll address that too. Let’s address these one at a time.  First, Carter mentions that in some scenarios there is a need to rehydrate data in order to back it up.  The process of rehydrating data may not require that the array have the physical capacity to store the data before it is backed up, but the array will require the CPU resources, I/O resources, bandwidth and time to rehydrate to data to back it up.  George goes on to say that this situation is “ugly, but not that ugly”.  I will tell you any time you put more resource requirements on systems that do backups, your running the risk that backups won’t get done.  One of the greatest challenges in IT is backup.  Backup administrators are running into backup window problems all the time.  Data is growing not shrinking; having to do more work on more data in order to protect it is a recipe for failure.  In my previous comments I may have incorrectly stated you need more disk space to do the backups, but I did correctly state that the array will require more system resources.  And where do these resources come from?  When the system is idle?  When is your storage array idle? Now, what if all you had to do was – well nothing.  Storwize sits in front of primary storage and stores your data, compressed, in real-time with no performance impact and preserving the envelope of the data file.  Then when it comes time to backup, the backup administrator does absolutely nothing different that he/she did yesterday.  Same shares are backed up, same clients, and all the work is done by the Storwize appliance, there is no load on the filer.  The next question is can Storwize keep up with the backup stream and the answer is YES.  As you saw in the Wikibon CORE blog, our time to compress is on the order of magnitude of milliseconds – the time to decompress is even less.  (I should also mention one thing Carter failed to mention, in order for backups to come off their system ‘transparently’ you need a software agent on the client – who wants to manage more clients?

Performance

Efficiency

Carter also states that, “It should be noted, though, that Data Domain results will be slightly worse with either Storewize or Ocarina compression – because compressing data makes it harder to find duplicates.” Now I hate to argue theory and I have been speaking about the advantages of random access compression, specifically with deduplication so I guess now I will show the results we have from live testing.  The fact of the matter is, Storwize makes deduplication better.  Because Storwize stores data in random access nature inside the compressed file, just like a regular file, the deduplication ratios are preserved and are identical to non-compressed files.  But don’t take my word for it, the chart on the left shows we made Data Domain faster and more efficient by pushing less raw data to the device.   The chart on the right shows that we made Data Domain more efficient in its optimization (good for the customer).  I would like to see the same charts from other vendors. Two other notes to mention.  First, if you consider snapshots a method of backup, then you need to pay special attention to your snapshot process with post process optimization solutions.  When using a post process optimization solution users MUST wait for the optimization process to complete before creating a shapshot or you will snap an un-optimized file which will cause you not to save space.  Additionally, when you backup a snapshot, you need to restore the entire snapshot in order to restore a single file.  Because the Storwize solution is in-band, users can perform snapshots during the compression process and all files will be optimized because Storwize writes all files in the compressed format. The second thing to consider is backing up deduplicated data.  In a NAS environment, if the primary storage array is doing the actual deduplication, all data has to be backed up in its full un-optimized form using NDMP (even if it will be deduplicated on the secondary device, if the devices don’t use the same deduplication algorithms, then the data must be rehydrated).  This increases IO and load on the filer substantially.  The same situation occurs with an external solution, however the rehydration may take place in a different location, but still impact the overall backup performance.  Additionally, it is important to remember that a deduplication process creates dependencies between files/blocks/objects which the storage is not aware of.  Backing up this data thru NDMP forces external deduplication solution to be involved in the backup process which significantly complicates backup administration and  more importantly adds risk and too many elements of your backup and restore processes. So, the bottom line, if the storage array is deduplicating the data, you backup the unoptimized/undeduplicated data, which can still be deduplicated on the backup target, but you are not getting any benefits in terms of shorter backups.  If an external solution performs the deduplication, then that solution is added into the mix of storage, external_dedupe, backup_software, backup_target, - this just sounds like a backup bouillabaisse that wouldn’t sit well with any backup administrator. Finally, and perhaps the most common backup for enterprise NAS systems is NDMP.  Ocarina will talk about a ‘dedupe aware’ NDMP – is this some special NDMP that they wrote?  Does it work for all files? Do you use it only for the Ocarina data and then the standard NDMP for non-Ocarina data?  This again is just adding complexity. The value of Storwize – transparency.  Install Storwize and EVERYTHING from your applications to your backup work EXACTLY as they did before – just faster – with nothing else to install – that seems pretty ‘effective’ and very simple to me. Look, at the end of the day technology solutions are as much about solving the business issue as it is about the end user.  Great technology that solves a problem but has impact to end users or to IT process isn’t a solution at all.  I have spent 10+ years of my career in customer support, 2+ years running professional services, 5+ year evangelizing products with customer input and feedback and 4+ years as an Analyst writing about technology for end users.  I can tell you with 100% certainty that every customer environment is different and it is why many solutions exist.  I will also tell you that I choose to work for companies that have solutions that are next generation, evolutionary and add real value to the end user.  Steve said it best when he talked about the Virtual System Administrator.  Their job is hard, backup is hard, let’s not make it any harder.  I fully admit I don’t know everyone’s solutions as well as I know the ones I represent, but I did choose the ones I represent for a reason and after a good deal of due diligence and Storwize not only adds technical value, saves customers a lot of money, it seamlessly and transparently fits into your storage environment with no impact to performance, end users or process. That is real value!