There was a great post a couple of weeks ago, with Tom Coughlin as a contributing editor, on Forbes’ news site about the floods that hit Thailand and how it will affect the disk drive market.  The great thing about the article is it truly highlights that necessity is the mother of invention.  What do I mean by that?  Over the past few “storage efficiency” has been a big topic with vendors.  Helping customers “do more with less”, especially in these stringent economic times, is key to the vitality of a number of businesses.  Technologies such as storage virtualization and thin provisioning have helped customers to slow their storage spend and get better utilization out of their existing storage.  Once customers have moved their utilization rates from 35% to 65% or 70%, time comes when new storage needs to be acquired to keep up with the growth of data.  The issue comes when there are no more disk drives to be acquired.  Due to the floods in Thailand, analysts predict that the storage industry could be 50 to 60 million units shy of the demand this quarter.  This does two things:

1)      Drives the price of disk higher, at a time when the expectation is to spend less for disk

2)      Has IT getting more creative on how they use and deploy their storage

It is the later that I want to focus on as paying more for disk is not necessarily the best option.  It is important to note that data grows for one reason, business does not stop, it needs to keep going and it is what is driving the demand on the data.

In the Forbes piece Tom talks about “a surge in new technologies because of this disk shortage” but he doesn’t cover some of the most innovative technologies that are available to help customers.  I would agree with Tom that we “could” see a surge in SSD but that would be short lived do to both supply and cost as well as a surge in tape, but these aren’t really “new technologies”.

New technologies for primary storage optimization can and will play a key role in helping IT be more productive with their existing capacity.  New technology such as Real-time Compression can help customers get back up to 80% of their existing storage capacity without losing any of their current capabilities or changing any of their data management processes.  The technology seamlessly integrates into your storage environment and compresses your data 50% to 80% (depending upon data type).  It also fits into IT’s existing data management practices without having to change anything.  No change is required to any of the applications, snapshots stay the same, replication stays the same even backup works without having to change anything in the environment.  And while some vendors may say “you can’t deduplicate compressed data”, you actually can deduplicate data written with Real-time Compression.

The Real-time Compression technology is truly a “new” technology that can expect to surge in this environment.  IT can deploy this technology and expect:

1)      Up to 80% compression on their primary storage

1. This means they can defer adding new capacity until the HDD market comes back and disk prices stabilize

2)      See up to 80% optimization in each of their downstream processes that use disk

1. Meaning up to 80% less capacity for snapshots
2. Meaning up to 80% less capacity for replication
3. Meaning up to 80% less capacity for backups

(In each of these cases, each process uses disk so there is a tremendous savings by just compressing the primary copy of the data)

3)      The technology will be transparent to their existing infrastructure

In addition, Real-time Compression can cut your cost per TB by a factor of your compression ratio (50% compression is a 2:1 cost reduction in your $/TB cost).  It is also the case, if you are looking to SSD for performance, you can now afford to spend some money on SSD or more money on SSD given the new cost model.

Now, the “new” technology does need to be efficient and fit into a customer’s existing infrastructure seamlessly or it isn’t really useful.  Asking IT to change their processes can be just as costly as purchasing new capacity in the long run.  I mention this because in a related story, NetApp is also fearful about what the HDD shortage will do for their business.  I find this ironic.  On a recent panel I was on at SNW with Larry Freeman of NetApp, he told the audience that NetApp filers have these “new” technologies “built in” to their WAFL file system, in fact they have 10 “storage efficiency” features built in to WAFL.  He went on to say that on a weekly basis they get reports from a number of systems in the field that “report in” on how customers are using their systems.  On average customers use only 3 of the 10 features.  When we polled the audience to ask them why, they said that while the feature may help them save space, they impact other areas of their operation.  Maybe it impacts system performance, maybe it impacts backup so they can’t use the feature.

The moral of the story is that I do believe that new technologies are going to “surge” (as Tom states) in his piece, because IT will need other alternatives to the shortage of disk drive that are available and the higher prices.  In addition, this will force IT to look at their environment to identify how to be more efficient with their storage environment as stuff like the flood could come up again and affect the supply and demand of HDD.  But the right technologies that not only help with storage capacity as well as data growth needs to be the answer to the challenge.  The best technologies fit into IT’s existing infrastructure and makes it more efficient overall.

Some interesting data came out of this panel.  There were probably over 150 people in the audience.  It was a well-attended session.  Also, Dave is VERY good about asking the audience questions.  Let me start by making sure we all know where everyone sits at the “storage efficiency table” that was on the panel.

• Larry Freeman is from NetApp – they claim, and I believe them, that they have 10 storage efficiency technologies that are embedded into WAFL
• Craig Nunes main focus on the panel was ‘zero reclamation’ to optimize storage
• I have a Real-time Compression drum I am beating
• Jarred Floyed focuses on data deduplication

Here are some questions and answers Dave got when speaking to the audience:

One last comment too that Larry Freeman stated was that NetApp has 10 storage optimization / efficiency technologies embedded into WAFL.  The interesting thing is that NetApp gets a report from 150,000 systems that "report in" over the weekend and they have collected statistics that users only use 3.2 out of these 10 efficiency technologies on average.

When the question was posed "why" is it that you have technology, that is “free” (more on that later) that can help you better optimize your storage, why they don’t turn it on?  To which the end user response was that they didn't want any change in the infrastructure that could require a change in their processes.  (This means that transparency is another thing that an appliance has to ensure it has.)

I also want to publicly commend Larry for a good deal of his comments and his honesty.  See Larry and I are technologists and sometimes we see the marketing arms of our companies sometimes stretch the truth to get what they want in the market by way of some FUD.  My example is this.  The RtC technology does provide high compression, for the life of the file without performance degradation.  Well, as you can see, this is what customers want give the response from the audience.  However, our friends at NetApp will say “yes, but our compression is free” (like that is supposed to be better).  Anyway, Larry addressed this in the same way I do.  He said that “you don’t get anything for free and there are tradeoffs, specifically in this case around performance”.  Now, I am not saying this to say “see, I told you so” I am saying it because we, as technologists, want to give the user the best answer.  I also admitted that an embedded strategy for technology is the right way to go, IF you can accomplish what the end users’ needs are, performance in this case.

Look, there is a reason why startups exist and why appliances exist.  Storwize didn’t invent compression; they invented a real-time platform that allowed compression to happen in real time.  We do “compression off load”.  We take the work off the array and put it on the appliance – how could that not be faster.  And yes, one day it will be embedded and there will be some other new great optimization technology that will start as an appliance and slowly find its way into the array.  It is the evolution of technology.

At the end of the day, here is what I have to say.  Every end user buys their storage for a reason.  It may be performance; it may be capacity who knows.  What I ask each user to do is to consider their BUSINESS needs for their storage and apply the RIGHT storage optimization technologies for their given environments.  There is not a “one size fits all” approach (this is why NetApp customers don’t choose to turn on all 10 optimization / efficiency technologies) but there are a handful that can help you get the most out of your storage.

Alright, landed safe in Prague and was picked up by one of my colleagues and whisked away to the IBM office.  There we did an interview with Czech writer Martin Noska from Computerworld for IDG in Czech Republic.  The first Noska informed me was that IBM is the number one in storage sales in Czech Republic (just like Poland!).  He also had some very good questions and he with “What are IBM’s biggest challenges in the storage business”?  I had thought about this for a while and I would have to say it is really about marketing our storage “solutions” to the customer base.  IBM is a double edge sword.  IBM is so big and has so many products it becomes difficult to market or message all of our products without inundating all of our customers and confusing them.  If you think about it, IBM has hundreds of thousands of customers and business partners, if not more.  This is one of our strengths.  When customers have needs or requirements we have very good input into our product portfolio, perhaps the best in the business.  Combine this with the fact that IBM has not only storage solutions but technology across the entire stack from servers to networking.  So when it comes to developing the right technology, that solves real customer problems, I would argue that IBM’s portfolio is the best in the business.  IBM takes an extreme amount of care when developing a solution to ensure that it matches the customer requirements based on the changing needs of IT.  Having an integrated portfolio that works well with our ISV partners, VMware for example, allows us to help customers speed their time to ROI and be very competitive in the market place.  The challenge is, how do we properly message our new solutions to our customers, in a timely manner so that they are well aware of new products without giving them too much information such that it just becomes noise?  It is difficult to say the least.

The interview went very well.  There were questions about tape, where we discussed the advantages of IBM’s LFTS technology for more advanced tape usage, we discussed the direction data deduplication will go as well.  Noska’s view was that there hadn’t been any advancement in data deduplication in the last 5 years.  I told him that for secondary storage, backup, that he is right, I also told him that the real advancement to deduplication will come when it is ready for primary storage.  Today deduplication isn’t ready for primary, but it will be soon.

On Monday the 13^th we traveled to visit Avnet.  They are a great IBM partner.  Like most partners they have a very large SMB install base and also like a lot of SMB feedback I have been getting, they are looking for a building block solution that has all of the software features implemented as a part of the stack.  SMB and Enterprise alike are starting to realize that the value in any array is becoming the software stack that makes the hardware, efficient, optimized, flexible, and dynamic.  IT’s job continues to get more and more challenging with developing strategic initiatives for the business to make them more competitive and it is the job of the vendor to make sure these solutions are as optimized and cost effective as possible.

We also visited DHL.  These guys have one of the greatest datacenters I have ever visited.  They are very advanced and push a lot of data.  The do some very strategic logistics for a number of companies in Europe and Asia.  They, like many others have a number of challenges.  Since my blog post about “The 5 Most Interesting things at VMworld” (#4) I heard something very interesting today.  I asked “What is your most challenging storage issue”?  He told me that storage was not is “most difficult” challenge.  Storage efficiency was important to him in order to keep driving down costs for his organization as they deliver a service to the different groups that make up DHL, but his most difficult challenge was with server I/O in his VMware environment.  If you read #4 in my post, regarding Proximal Data, this is exactly the issue the address.  As VM instances grow on the physical servers, the I/O starts to become the big problem.  DHL runs over 4000 instances of VMware and as the business demands more applications and application resources, they are bound by the I/O of the server, which also causes them to WAY over provision their storage for performance reasons.  This is very time consuming, management intensive and expensive.  The combination of a solution like Proximal Data as well as compression can help them optimize their infrastructure to save money and deliver better, more cost effective services to their lines of business.

On the lighter side, I spend the weekend in Prague.  What an amazing city.  The weather was fantastic and I was able to take a lot of great photos.  I walked around Prague Castle, ate some authentic Czech food, visited the memorial for the Czech hockey players that passed in the Russian plane crash and met some pretty interesting people.  You can check out some of my photos of Prague at www.facebook.com/skenniston.  Coincidentally the photo above shows the "Golden Lane" where the Alchemists worked to turn anything they could find into gold in the city of Prague.

After landing in Warsaw, I got into a car with the local sales leader for Poland and we drove to the event location.  It was a 2 hour drive.  First, the roads and the land in Poland reminded me very much of my home time in Maine.  Very scenic and rural but beautiful and peaceful.  We talked storage for 2 hours and I am always festinated at the thirst for knowledge there is when I travel.  It was a great ride followed up by a customer reception and some local Polish brew.

Thursday I spent the day in Sterdyn, Poland for IBM Storage University.  There were 30 customers at the event and it went very very well.  The event was at Palac Ossolinski, today used as an event center but has a very rich history, in fact at one point it was used as a medical facility in WWII.  The photo is of the building where we had the event.  The topics we covered were:

• Storage Efficiency
• EasyTier
• ProtecTIER
• XIV
• Real-time Compression
• V7000

The customers were very interactive and provided a lot of insight to their environments.  Interestingly enough I learned during our customer reception that IBM storage is #1 in Poland with HP second and EMC third.  This is a true testament to the IBM sellers and the customers who use the IBM products every day to drive their business.  I also learned that the data break down in Poland is 90% block, 10% file which I found interesting and would be interested to check back 12 months from today to see how it will be different.

I did learn something very interesting in Poland.  The question was asked “Why XIV”?  What is so special about XIV.  The answer was awesome.  The answer started with 2 questions:

1)      How old is RAID?

2)      How old is your iPhone?

The reality is data growth is out pacing what traditional RAID can handle and data profiles are changing as well.  These combined have driven new technologies like Cleversafe, Cloud Computing, Hadoop and XIV.  Just like the iPhone is a new approach to the smart phone based on new things we know about how these smart phones are being used, we know more about how data and storage is being used.  New ways to deliver capacity and performance are needed in order to keep up with the changing times.  I thought it was a very good answer in terms that make customers think.

Thursday evening I traveled back to Warsaw where I got in a bit late and just went to a local pub, Sketch.  Grabbed a small bite and some local mead and then headed back to the hotel.  I did get to see the local Palace of Culture and Science in the middle of Warsaw, very impressive, built as a gift from Russia to Poland.

I have an early flight to Prague.  I am very excited about this part of the journey as I have always wanted to travel to Prague.  Press meeting right when I land.  Stay tuned.

The first city on my Eastern European trip was Moscow.  I think the traffic here is worse than the 101 in Silicon Valley during the dot com era.  That said, it was a great visit.  I spoke at the Information Infrastructure Conference at the Swissotel convention center in Moscow.  It was the first time I spoke to a group of people with an interpreter.  It was like being at the UN.  The two main topics were Storage Efficiency and Real-time Compression.

I spoke with a few customers and the press and in dealing with the data growth challenges they wanted to know, “When it comes to big data, what is next, is it ‘huge data’”?  Data growth clearly a concern.  Interesting enough though most of the questions, came around my title of “Evangelist”.  One report told me, “if an Evangelist is ‘preaching the word of storage’ then why not just call yourself an Apostle”?  How do you think that would look on an IBM business card: Global Storage Efficiency Apostle?

The next day I did a day of “sales enablement” in the Moscow office.  We discussed mostly how to sell and position Real-time Compression and what is next for the technology.  I was very impressed with the team.  They were very technical and knew quite a bit about Real-time Compression and really wanted to know in more detail how the technology was invented.  This means they are really talking about the technology and the customers are drilling down into the next level of detail.  There are a lot of good opportunities for the technology in Moscow and I look forward to hearing more about the success of Real-time Compression there.

I didn’t have a lot of time to sight see but I did make it to Red Square.  You can actually buy a beer outside in Red Square and walk around.  So I did.  I took a few photos and then as the US was getting going, I had some work calls to attend to.  That evening I spent on the 34^th floor of my hotel having dinner.  It was a great view of Moscow.  I hope to come back.

Next stop, Warsaw Poland.  Stay tuned.

Today I begin a 12 day trip to Easter Europe to talk about IBM Storage.

The trip will take me to:

• Moscow, Russia
• Warsaw, Poland
• Prague, Czech Republic
• Ljubljana, Slovenia
• Umag, Croatia

In Russia, on September 6, I will be at the Information Infrastructure Conference and the following day meeting with customers to discuss storage and storage efficiency.

In Poland on September 8, I will be presenting IBM’s Real-time Compression at Storage University.

In Prague I will be meeting with the press as well as speaking with customers.  Additionally, I will be spending the weekend in Prague, a city I have always wanted to visit.

In Slovenia on September 14, I will be presenting at IBM’s Innovation Center at an IBM Solutions Event.

Finally in Croatia on September 15, I will be at the IBM Forum, the largest IBM even in Croatia.

In each location, I will be speaking with partners and customer on IBM’s innovation in storage, storage efficiency and Real-time Compression.  I am looking forward to learning what the largest storage challenges are across Eastern Europe and users go about solving their challenges.  Additionally, I will be doing some local enablement for our partners and sellers.

I will blog from each location.  I will talk about the professional part of my travels as well as, hopefully, one personal event.  I have tried to make sure that in each city I have time to do one interesting thing.  I don’t know when, if ever, I’ll be back to these cities and these are some places I have always hoped to go.  Too often we travel and its all business.

Also stay tuned, when I land I will have an update from my trip to VMworld.  It was fantastic.  Truly the best end user show around.  I learned a great deal and can’t wait to share some of what I saw.  As always – comments are always welcome.

This weekend I had an interesting conversation with my Dad.  We were discussing backup.  My dad basically runs IT for the State of Maine.  The State of Maine uses CommVault backup software.  So I posed the question to him, “What would it take for you to rip out CommVault and replace it with another solution.  He thought about it for a moment and replied “I wouldn’t”.  His answer came down to a couple of reasons.

First was the expense.  It’s not just about buying the new software, it would be training people to run the new software and it would be about throwing away the massive investment they have in their existing product as well as converting all the years of backup takes created with one software to the new software.  This is one of the biggest things vendors forget when trying to sell a customer on their backup software.

Second was the fact that, feature for feature, the top 5 traditional backup software products are not really that different from one another.  Sure, I do agree that some products have features that others don’t, and others products have features that work better than others, but in reality, the delta is so small and the workarounds are so simple it doesn’t really matter.  Unless your replacing traditional backup software with an evolutionary source based data deduplication software (which is only applicable for some environments) there is no advantage to switching software.

The challenge is if Data Protection is still one of the biggest and most expensive pain points within IT, how do the problems get resolved if replacing the software controlling it all is too costly to change?

It’s time to start asking new questions about how to solve the problem.  In the backup ecosystem there are a few things IT has to juggle in order to maintain equilibrium.  First there is data growth.  As data continues to grow, it gets more challenging to complete backups in the time available.  Second is availability, this is where the terms RPO and RTO (recovery time, recovery point objectives) come into play.  How quickly can I get my data back, and how old is it (how much data have I lost) when I do get it back.  Third is corporate governance.  Keeping the business out of jail, or from losing a law suit, is a key role of data protection and when done properly can actually help with backup.

So when trying to balance all of this, where does one start?  It starts with one key concept, What is the value of the data to the business at any given point in time? Based on this, IT can begin to develop a very effective data protection strategy while keeping the cost of doing so in check.

There are two things that must happen in the development of this strategy.  First IT needs to answer the data value question.  This can ONLY be done by having conversations with lines of business managers to understand a few key metrics.  Metrics such as:

1)      How much does it cost the business to be down?

2)      How much does it cost when data is lost?

3)      Does the company need to adhere to any regulatory requirements?

There are many other questions, that need to be answered but these are a few of what IT needs to be thinking about.  Next, IT needs to establish some goals.  These goals need to be realistic given the budget and they need to align to the value of the data.  In other words, apply the right technology to the right data based on its value to achieve balance.  These goals should take into consideration:

1)      How do I ensure the fastest recoverability reasonable for a given data set?

2)      How do I move as little data as possible (moving data is expensive)?

3)      How do I recover as little data as necessary (smaller amounts of data being recovered = faster recoveries)?

4)      How do I ensure I store data on the lowest cost medium practical to meet my SLAs?

5)      How do I delete data at the right time?

These are very lofty goals.  Keep in mind, unless your company is in the business of providing data protection services, your company doesn’t make any money backing data up.  Backup is an insurance policy.  The trick is to balance the costs of protecting that information with the value of the information.

Where does IT start?  Start with the goals.  If RPOs/RTOs are the challenge, try leveraging space efficient snapshots to have data local and more point in time copies for faster, “newer” recoverable data.  If backups need to be faster to help close the backup window, backup to disk, then copy that data to tape.  If that disk is too expensive, deduplicate the data.  To move as little data as possible and recover as little data as possible, leverage real-time compression on your primary storage.  By compressing data as much as 5x, AND ensuring that the compression solution works with existing data protection processes, there will be a significant reduction in the amount of data that needs to be moved and stored in the backups giving back a great deal of time in the backup window.  Next, think about archiving technologies.  Most data retention practices have a schema similar to 14 dailies, 4 weeklies, 11 monthlies and depending on the yearly retention practice X number of yearly’s.   A proper archiving technology can take the multiple copies of data being stored in the “yearly” data set and reduce it significantly.  By taking that data out of the backup flow, again, the costs of disk and tape storage go down and moving less data gives back time in the backup window.  Also, setting the proper archiving policies will identify when to delete data which saves on storage costs, both the medium and the footprint (tapes at an offsite facility).

At the end of the day, there are a number of solutions available to customers that can have a profound impact on your business without having to rip out your existing backup software that don’t cost a lot of money and won’t impact the existing investment.  I encourage data protection professionals to work with vendors who take a more consultative approach in helping to solve the data protection problem.  Work with a vendor that has a very large install base who has implemented a number of data protection solutions and can draw on a wealth of use cases and best practices collected over time.  Solving the data protection challenge is not easy, to solve it properly, you need to properly align the value of your data to the technology used to protect it and most of all it takes a good deal of experience reach out to the pros.

We all know compression is not new and it is coming as a standard feature in a number of storage systems. The issue is, each of these technologies has a significant impact on performance - both primary storage performance as well as the performance on all of the back end operations such as backups, replication etc...

IBM's Real-time Compression doesn't have any of these limitations - listen to Ed to hear more.

Storage is elastic.  How do I know you ask?  Yesterday I visited a customer who is using the Storwize product to do Real-time Compression on their primary storage.  The customer is Allianz and has been using the product for over a year.  They see 75% compression on their users home directory data.  To give you an idea, Allianz is an insurance company and generates TONS of spreadsheets, 14TB worth of spreadsheets (okay, not all 14TB is spreadsheets but you get the picture).

Prior to Allianz purchasing the Storwize technology, Allianz didn’t have great data management practices.  Users store data in their home directories and there is really no discipline around deleting or cleaning up files so data just grows.  Additionally, storage isn’t really budgeted for.  Overall IT is but at a storage level, they just purchase some when the need some.

Again, prior to the Storwize technology, Allianz had their primary storage and a backup to tape at their local site.  They then replicated the data to their remote site and also performed a backup to tape.

Allianz has an overall IT mission to reduce spend by 10% per year.  The thing to think about is that this 10% could come from a lot of places including data management.

Once the Storwize technology was installed the first things they saw were:

• 75% capacity optimization
• Better data management capabilities through Storwize reporting
• The ability to keep more data on line and available for faster recoveries
• No change in any of their existing storage processes

Storwize allowed Allianz to compress their primary storage 75% and then leverage the snapshot feature on their NTAP to create compressed snapshots seamlessly for quick recovery.  Additionally, Allianz now does local backups to disk instead of tape reducing the amount of time it takes to do local backups and providing them with on line backup data for much faster recoveries which is much easier to manage.  Allianz then replicates 75% less data on a daily basis to their remote site which saves on replication costs, then at the remote site they are able to perform the same snapshots and backup to disk that they do at the local site in a very timely manner providing them with a highly efficient DR site.  Then they create one backup to tape to archive their data.  It is important to note that the Storwize technology fit into their environment with no re-architecture to any of their applications, servers, networks or storage nor did any of their IT processes change when it came to backups and recoveries - they worked seamlessly with the compressed data as if it were non-compressed data.

The point of the piece well is two fold.  One, customers are using the Storwize technology to successfully manage their storage growth and drive down costs.  By performing more snapshots and backups to disk they are able to perform faster recoveries which save them time, make them more competitive by being able to respond faster and they haven’t had to purchase storage for two years.

The second point is that disk is elastic.  The more space you give users, it is proven that they will just use the space.  Now instead of one local copy and a backup to tape, Allianz uses the extra disk space to store more data that allows them to be much more effective at business continuity.  Give users the space, and they will find a way to use it.

This was indeed true when data deduplication became popular.  Every vendor I spoke with five years ago that sold disk for backups claimed they would sell less disk for backups.  The reality was because it saved time in the backup process customers did more backups to disk.  Because it reduced the cost of disk and brought it in line with tape – customers stored data on disk vs. tape.  Because it could replicate data very efficiently customers bought more disk for a second site to keep data in a DR site.  If a customer would have bough 10TB of disk for disk based backups before, they still bought 10TB and kept more backup data on-line for a greater variation of recovery point objectives.  All viable business cases.  Now if a customer isn’t doing data deduplication – well they are one pink slip away from a deduplicated recovery from disk.

Customers use the disk to their advantage which goes to show that if a customer has a budget of $1M they will spend the $1M – how it gets divided up may be different.  For example in the Storwize case, instead of buying $1M of disk, customers now buy $700K of disk and $300K of Storwize and use that disk 5 times more efficiently.

As for Allianz, Daniel Gill said “Storwize was a no brainer.”  Not only does it help him save his company money, it makes them a much better IT shop by having more data readily available when their users need it.