There was a great post a couple of weeks ago, with Tom Coughlin as a contributing editor, on Forbes’ news site about the floods that hit Thailand and how it will affect the disk drive market.  The great thing about the article is it truly highlights that necessity is the mother of invention.  What do I mean by that?  Over the past few “storage efficiency” has been a big topic with vendors.  Helping customers “do more with less”, especially in these stringent economic times, is key to the vitality of a number of businesses.  Technologies such as storage virtualization and thin provisioning have helped customers to slow their storage spend and get better utilization out of their existing storage.  Once customers have moved their utilization rates from 35% to 65% or 70%, time comes when new storage needs to be acquired to keep up with the growth of data.  The issue comes when there are no more disk drives to be acquired.  Due to the floods in Thailand, analysts predict that the storage industry could be 50 to 60 million units shy of the demand this quarter.  This does two things:

1)      Drives the price of disk higher, at a time when the expectation is to spend less for disk

2)      Has IT getting more creative on how they use and deploy their storage

It is the later that I want to focus on as paying more for disk is not necessarily the best option.  It is important to note that data grows for one reason, business does not stop, it needs to keep going and it is what is driving the demand on the data.

In the Forbes piece Tom talks about “a surge in new technologies because of this disk shortage” but he doesn’t cover some of the most innovative technologies that are available to help customers.  I would agree with Tom that we “could” see a surge in SSD but that would be short lived do to both supply and cost as well as a surge in tape, but these aren’t really “new technologies”.

New technologies for primary storage optimization can and will play a key role in helping IT be more productive with their existing capacity.  New technology such as Real-time Compression can help customers get back up to 80% of their existing storage capacity without losing any of their current capabilities or changing any of their data management processes.  The technology seamlessly integrates into your storage environment and compresses your data 50% to 80% (depending upon data type).  It also fits into IT’s existing data management practices without having to change anything.  No change is required to any of the applications, snapshots stay the same, replication stays the same even backup works without having to change anything in the environment.  And while some vendors may say “you can’t deduplicate compressed data”, you actually can deduplicate data written with Real-time Compression.

The Real-time Compression technology is truly a “new” technology that can expect to surge in this environment.  IT can deploy this technology and expect:

1)      Up to 80% compression on their primary storage

1. This means they can defer adding new capacity until the HDD market comes back and disk prices stabilize

2)      See up to 80% optimization in each of their downstream processes that use disk

1. Meaning up to 80% less capacity for snapshots
2. Meaning up to 80% less capacity for replication
3. Meaning up to 80% less capacity for backups

(In each of these cases, each process uses disk so there is a tremendous savings by just compressing the primary copy of the data)

3)      The technology will be transparent to their existing infrastructure

In addition, Real-time Compression can cut your cost per TB by a factor of your compression ratio (50% compression is a 2:1 cost reduction in your $/TB cost).  It is also the case, if you are looking to SSD for performance, you can now afford to spend some money on SSD or more money on SSD given the new cost model.

Now, the “new” technology does need to be efficient and fit into a customer’s existing infrastructure seamlessly or it isn’t really useful.  Asking IT to change their processes can be just as costly as purchasing new capacity in the long run.  I mention this because in a related story, NetApp is also fearful about what the HDD shortage will do for their business.  I find this ironic.  On a recent panel I was on at SNW with Larry Freeman of NetApp, he told the audience that NetApp filers have these “new” technologies “built in” to their WAFL file system, in fact they have 10 “storage efficiency” features built in to WAFL.  He went on to say that on a weekly basis they get reports from a number of systems in the field that “report in” on how customers are using their systems.  On average customers use only 3 of the 10 features.  When we polled the audience to ask them why, they said that while the feature may help them save space, they impact other areas of their operation.  Maybe it impacts system performance, maybe it impacts backup so they can’t use the feature.

The moral of the story is that I do believe that new technologies are going to “surge” (as Tom states) in his piece, because IT will need other alternatives to the shortage of disk drive that are available and the higher prices.  In addition, this will force IT to look at their environment to identify how to be more efficient with their storage environment as stuff like the flood could come up again and affect the supply and demand of HDD.  But the right technologies that not only help with storage capacity as well as data growth needs to be the answer to the challenge.  The best technologies fit into IT’s existing infrastructure and makes it more efficient overall.

Some interesting data came out of this panel.  There were probably over 150 people in the audience.  It was a well-attended session.  Also, Dave is VERY good about asking the audience questions.  Let me start by making sure we all know where everyone sits at the “storage efficiency table” that was on the panel.

• Larry Freeman is from NetApp – they claim, and I believe them, that they have 10 storage efficiency technologies that are embedded into WAFL
• Craig Nunes main focus on the panel was ‘zero reclamation’ to optimize storage
• I have a Real-time Compression drum I am beating
• Jarred Floyed focuses on data deduplication

Here are some questions and answers Dave got when speaking to the audience:

One last comment too that Larry Freeman stated was that NetApp has 10 storage optimization / efficiency technologies embedded into WAFL.  The interesting thing is that NetApp gets a report from 150,000 systems that "report in" over the weekend and they have collected statistics that users only use 3.2 out of these 10 efficiency technologies on average.

When the question was posed "why" is it that you have technology, that is “free” (more on that later) that can help you better optimize your storage, why they don’t turn it on?  To which the end user response was that they didn't want any change in the infrastructure that could require a change in their processes.  (This means that transparency is another thing that an appliance has to ensure it has.)

I also want to publicly commend Larry for a good deal of his comments and his honesty.  See Larry and I are technologists and sometimes we see the marketing arms of our companies sometimes stretch the truth to get what they want in the market by way of some FUD.  My example is this.  The RtC technology does provide high compression, for the life of the file without performance degradation.  Well, as you can see, this is what customers want give the response from the audience.  However, our friends at NetApp will say “yes, but our compression is free” (like that is supposed to be better).  Anyway, Larry addressed this in the same way I do.  He said that “you don’t get anything for free and there are tradeoffs, specifically in this case around performance”.  Now, I am not saying this to say “see, I told you so” I am saying it because we, as technologists, want to give the user the best answer.  I also admitted that an embedded strategy for technology is the right way to go, IF you can accomplish what the end users’ needs are, performance in this case.

Look, there is a reason why startups exist and why appliances exist.  Storwize didn’t invent compression; they invented a real-time platform that allowed compression to happen in real time.  We do “compression off load”.  We take the work off the array and put it on the appliance – how could that not be faster.  And yes, one day it will be embedded and there will be some other new great optimization technology that will start as an appliance and slowly find its way into the array.  It is the evolution of technology.

At the end of the day, here is what I have to say.  Every end user buys their storage for a reason.  It may be performance; it may be capacity who knows.  What I ask each user to do is to consider their BUSINESS needs for their storage and apply the RIGHT storage optimization technologies for their given environments.  There is not a “one size fits all” approach (this is why NetApp customers don’t choose to turn on all 10 optimization / efficiency technologies) but there are a handful that can help you get the most out of your storage.

Today I begin a 12 day trip to Easter Europe to talk about IBM Storage.

The trip will take me to:

• Moscow, Russia
• Warsaw, Poland
• Prague, Czech Republic
• Ljubljana, Slovenia
• Umag, Croatia

In Russia, on September 6, I will be at the Information Infrastructure Conference and the following day meeting with customers to discuss storage and storage efficiency.

In Poland on September 8, I will be presenting IBM’s Real-time Compression at Storage University.

In Prague I will be meeting with the press as well as speaking with customers.  Additionally, I will be spending the weekend in Prague, a city I have always wanted to visit.

In Slovenia on September 14, I will be presenting at IBM’s Innovation Center at an IBM Solutions Event.

Finally in Croatia on September 15, I will be at the IBM Forum, the largest IBM even in Croatia.

In each location, I will be speaking with partners and customer on IBM’s innovation in storage, storage efficiency and Real-time Compression.  I am looking forward to learning what the largest storage challenges are across Eastern Europe and users go about solving their challenges.  Additionally, I will be doing some local enablement for our partners and sellers.

I will blog from each location.  I will talk about the professional part of my travels as well as, hopefully, one personal event.  I have tried to make sure that in each city I have time to do one interesting thing.  I don’t know when, if ever, I’ll be back to these cities and these are some places I have always hoped to go.  Too often we travel and its all business.

Also stay tuned, when I land I will have an update from my trip to VMworld.  It was fantastic.  Truly the best end user show around.  I learned a great deal and can’t wait to share some of what I saw.  As always – comments are always welcome.

History truly does repeat itself.  We are talking about the history of data storage.  Every once and a while a new technology comes along that requires a new way to think about infrastructure.  Notice I said “infrastructure”.  I’d like to paint two analogies:

Analogy 1: RAID – Prior to RAID users stored their data on disk and if they could afford it, they backed that data up to have a protected copy of their data.  When RAID came out, users were able to store their data on multiple disks appearing as one device.  The benefits to this were, increased data reliability, better performance.  This new technology however, fundamentally changed how disk was sold, but the questions were the same:

1. How much capacity do you need?
2. What type of performance does your application require?

The sales reps point of view changed.  There were a number of new considerations that needed to be taken into account.  First, the age old question, “Will I sell less storage “stuff?”  Remember the person, at the time, selling the disk was probably also selling the backup tape and software to protect that information.  If the disks are more reliable, maybe the customer won’t need as much tape?  Second, when the capacity question came up, the seller also needed to know what type of RAID the customer wanted to ensure they sold them enough drives.  It was no longer as simple as asking the capacity requirements and dividing it by the drive capacity at the time.  Now depending upon RAID levels there was a new set of math that needed to be done.  Third was the notion of performance and more spindles meant more performance so now that the capacity equation was solved for, you also needed to know the I/O requirements in order to make sure the right number of drives were sold to solve for the capacity as well as the performance.

Guess what, we figured it out and the industry never looked back.  RAID is a defacto standard in all storage subsystems today, I even run RAID in my home.  The business benefits of having RAID far outweighed the costs.  In fact, it is probably one of the first times in storage history that the question of, “how can you afford not to have it”, came up.

Analogy 2: Virtual Machines – When VMware came out the value proposition was, do more work, with less physical infrastructure.  And again, the business benefits far outweighed the technology hurdle of implementing the new solution.

Keeping in mind that it is much harder to change process in IT than it is to change technology, IT decided that this new way of serving up processing power to applications was well worth all of the process changes that it would require.  One example, backup would need to change when implementing virtual server technology.  The data would grow 4x and the processing of that information for backup would take longer, in a world where time was all to valuable.  However the business benefit justified the change.

Again, the sellers questions were consistent:

1. How many virtual servers do you need? (Capacity)
2. What type of performance do you need for each virtual server?

The answers to these questions allowed a sales rep to configure the right number of physical systems to handle the right number of systems to make the line of business successful.  Additionally, some of the same considerations came up.  “Will I sell less server and make less money?”  Now that there was new server technology (more processors, the ability to handle more memory) systems could be bigger, and more expensive.  Sellers also needed to know a bit more about “capacity”, how many virtual systems could a physical system run successfully?  They also needed to have an understanding of performance.  Now sellers were configuring systems to run the equivalent of 20 to 100 servers on one system.

Today I would suggest that we are at a cross roads in history. New technology has come along that will have a significant impact on the storage world.  First, research from IBM reflects the fact that disk drives can no longer keep getting two times as dense for half the cost as they had been throughout the late 90’s and early 2000’s.  The technology doesn’t exist today to make the drives spin faster, stay cool and not loose data.  Until now.  Real-time compression is a game changing technology that will add significant value to the storage industry without having to change the way IT thinks about the deployment of their storage.

Data is growing at such a significant pace today and with the latest IBM research about disk capacities, something needs to change.  Data centers are just running out of space and more customers want to keep more data on line for reasons such as competitive edge or compliance, but no matter the reason, they want access to their information.  Enter real-time compression.  Now there is a fundamental difference between real-time compression and other compression technologies and compression implementations but I am not going get into it here, but it is safe to say that post process and in-line compression are very different than real-time compression and users can’t get the benefits of improved primary storage capacity, transparently, with no performance impact with anything but real-time compression technology.

Again, real-time compression, like other game changing technology, doesn’t require any new questions; there are just simply a new set of math equations.

1. How much capacity is required?
2. What is the performance requirement?

In time, real-time compression will be as ubiquitous as RAID, and just like users don’t think that much about RAID, users won’t need to think about compression.  Compression will become an expected feature of the array.  It doesn’t matter that it now takes fewer drives to satisfy the original question around capacity and performance.  With data growing as fast as it is and with disks not being able to keep up their growth pace, something needs to change and that something is real-time compression.  Soon, it won’t matter what the physical disk capacity is of a disk drive, it will be about a disks virtual disk capacity, what it has the capability of storing that matters.  It is time we all started thinking this way.

“Storage Efficiency” has become a big topic over the past 12 months.  There are a number of new technologies that have come out in the last few years that are helping to deal with storage growth.  We all know that data is the root of the decisions that drive business today.  The more data you have, hopefully, the better decisions you can make to drive your business to success.  The question is, “what is the value (and hence the cost) of the infrastructure to create that success?”  What we do know is that the ability to put more data in a highly efficient footprint can give your company a competitive edge.  There are five technologies that can help an IT organization create an efficient storage infrastructure.  These are:

1)      Tiering

2)      Virtualization

3)      Thin Provisioning

4)      Compression

5)      Deduplication

It is also important to point out that there are some semantics when talking about storage efficiency, specifically between efficiency and optimization technologies.  I think it is useful to attempt to define these as they lead us to picking the right solutions for what we are trying to accomplish.  For the purpose of this post, efficiency will relate to making existing capacity more useful and optimization will mean making more capacity out of existing capacity.

Using these definitions, technologies such as Tiering, Virtualization and Thin Provisioning are efficiency technologies.  These technologies help to utilize the existing capacity that you have.

Tiering is technology that is used on about 10% of your data or less.  It is used to move data that requires higher performance to flash storage.  Good tiering technology analyzes data access patterns and moves the most active data to the highest performing disk.  It doesn’t really change the amount of physical capacity that is required; it just changes what type of capacity is required and allows IT to make sure data is operating as fast and efficiently as possible.

Virtualization technology allows IT to make sure disk utilization is used as efficiently as possible.   Until recently storage utilization rates were around 50%.  By leveraging virtualization technology, IT can group pools of storage so they don’t need to purchase capacity needlessly.  Virtualization can be used on 50% to 60% of your storage but it doesn’t change your physical capacity infrastructure requirements and at most allows users to take advantage of 20% to 40% of their capacity that they once didn’t access.

Similar to Virtualization technology, thin provisioning technology also can be used on 50% to 60% of your capacity however, thin provisioning technology gives IT about 10% to 40% of their capacity back.  Thin Provisioning helps IT manage their existing capacity and their utilization by being able to make capacity available to users much easier again however it doesn’t change the amount of physical storage infrastructure required.

Optimization technologies help IT to better manage their physical storage footprint.  Optimization technologies optimize existing infrastructure by allowing users to put more capacity in the physical same space.  The two technologies that are currently used today are data deduplication and real-time compression.

Optimization technologies are a bit tricky.  There is a balance that is required between optimization and performance and availability.  At the end of the day, IT chooses the storage it buys with two very important characteristics in mind, performance and availability.  Optimization technologies can not affect these characteristics.  It is for this reason that data deduplication really isn’t ready for “prime time” on primary, active storage.  Data deduplication creates too much of a performance impact on primary, active data.  Today, data deduplication could be used on about 10% to 15% of the primary, less active capacity that is in the data center and only provides about 30% to 50% overall optimization.  In other words deduplication technology can impact the physical infrastructure by as much as 10%, meaning IT may not need to buy as much physical capacity.

Real-time compression, on the other hand, has one of the most dramatic affects on primary storage capacity.  Real-time compression can be used on as much as 85% of the storage footprint and can compress data between 50% and 80%.  That said Real-time compression could have IT purchase as much as 70% less overall storage capacity.  Real-time compression also does not affect the main characteristics for which users buy storage (performance and availability).  IT could have as much as 70% less footprint but keep the same amount of data or more on-line.  Additionally, IT can now purchase storage opportunistically without having to have such a dramatic impact on their infrastructure, process or budgets.  This allows companies to keep more capacity on line and available to help companies do more analytics on more capacity and become more competitive.

When deciding which storage efficiency technology will have a more effective impact on your overall environment and budget, start with optimization technologies and start to get the data growth under control.  Adding value to the line of business that can drive revenue with more data will make you a hero and your business more successful.

Great conversation on Linked In about deduplication and compression for storage efficiency in the Data Storage Professionals Group.  Help the storage community answer this question:

Does anyone has any experience in NAS de-duplication at filesystem level, like NetApps. Does it really work? I concerns/limitations?

Over the past few weeks I have witnessed the proverbial mudslinging that takes place in the blogosphere when marketing feathers are ruffled.  Most recently I was reading Rich Anderson of The StorageSavvy Blog.  The article was "Compression better than Dedup?  NetApp Confirms!"

I have to agree with Rich on many fronts.  First, "When all you have is a hammer, everything is a nail."  Rich points out vendors have to sell "what's in the bag" so it is conceivable that all problems look like they can be solved with their solution.  If you look back over the last few years NTAP has always had a "me too" reputation.  Whatever the industry has, they have one too and its better.  For the last few years, while competing against Storwize, they have pulled the EMC tactic of trying to stall a market by saying, "We have optimization for primary storage with deduplication."   The reality is, you can't use it in real time, it is a resource hog, and again Rich mentions, the only use case it works well on in primary storage is VMware (and that is ONLY IF the customer stores their data outside the .vmdk file otherwise compression is much better).  Now that NTAP has compression their story has changed saying that compression on primary storage is better for most use cases.  Duh!  The folks at Storwize (now IBM Real-time Compression) have been saying that for years.  Why, deduplication is great for repetitive data sets, i.e. backup, not primary storage.  There just isn't that much repetitive data in primary storage.  Again, NTAP is trying to stall the market saying they have "in-line" compression for primary storage.  Sorry guys, not good enough.  In-line is NOT Real-time.  Rich also points out that the key characteristics of storage for customers are capacity and performance.  Patrick Rogers of NTAP has said publically that compression WILL indeed impact performance and that they even have a tool that will tell you how much performance will be impacted.  While NTAP may say compression is "free", we all know nothing worth having in life is free, you get what you pay for.  If you need the performance to do compression you are going to have to perform a major upgrade to  your filer in order to just be able to perform compression let alone try to do compression in real time.  No real savings there.

It’s a simple fact that embedding optimization technology into the array you are optimizing is the right direction, it is just not ready for prime time today for a number of key reasons.  If you can't implement the technology in the array and preserve performance, availability, price and feature set, then its not a viable solution.  I know that every answer in IT is "It depends", and I am sure there are some use cases where the solution is "good enough" for some customers who meet all the proper criteria, the reality is, it is not ready for the general public.

On the other hand, if you can off load the work from the array in an appliance model, preserve performance and ensure 100% transparency to applications, infrastructure and processes then that is a real solution.   This is exactly what the real-time compression technology does from IBM.  Yes it is an appliance that sits in your CIFS/NFS storage infrastructure.  However, the benefits of the time based compression technology, along with the ability, by offloading the array from doing the compression, to maintain and in some cases increase performance, and the fact that putting the technology in place requires no change to your applications, no changes to your storage infrastructure and no changes to your downstream processes means this solution is the top primary storage optimization solution on the market.  It's not a big surprise.  First the technology comes from an Israeli startup, some of the smartest minds in storage / high tech.  Second, most great solutions start out as an appliance.  They are easy to deploy and allow you to get an understanding of how the technology really works in multiple use cases before trying to embed the solution with limited information.

Below are the top 10 reasons to deploy a Real-time Compression appliance for NAS from IBM over NTAP compression technology:

Of the customers running the technology in their environment over 57% of these customers have rated the Real-time Compression technology an "Excellent value – one of the top values in my data center" or better.

I am guessing by David Vaughn's visceral reaction that Rich hit a nerve that is more than just the typical old NTAP / EMC battling.  I don't think anyone ever told David that usually when you have such a reaction to something, you must be trying to hide something.  In this case, while NTAP has great products, they fall short when it comes to optimization technology.  Also, would you want your migration process to look like this?

In his post "NetApp Takes the "Primary" Lead for Data Reduction" - which seems more like theory and a commercial for NTAP than reality (see comments @ The Register) the SANMan states:

"Yes, Ocarina and Storwize have appliances that compress and uncompress data as it’s alternatively stored and read but what performance overhead do such technologies have when hundreds of end users concurrently access the same email attachment? As for Oracle’s Solaris ZFS file system sub level deduplication which is yet to see the light of day one wonders how much hot water it will get Oracle into should it turn out to be a direct rip off of the NetApp model."

I have two comments:

1) You are right - you CAN'T do deduplicaiton on primary if you affect performance.  All indications for customers are that they cannot use NTAP deduplicaiton or even compression 'in-line' as the performance is just too terrible so all processes must be done post-process.

2) I direct your attention to the Wikibon Blog on CORE - "Dedupe Rates Matter...Just Not as Much as You Think" - Storwize can do in-line data optimization without any performance degradation.  So the question is - if customers can 'Optimize without Compromise' - why wouldn't they?

Updated 6/7/2010 - Oh, quick question - how does the SANMan get away with the graphics he uses?  I would think that Walt Disney & Pixar would get a bit upset with the use of the character Carl Fredricksen, no?

I’m sure by now most of you all have read the fury lately over Tom Georgens’ comments about the future of storage tiering.  A number of folks who have ‘tiering’ technology reacted with disdain (see a list on Storagerap).  Some wondered how a storage visionary like Tom could turn his back on technology that helps people save money in storage.  Some even suggested that this is just marketing to overcome deficiency in the NetApp product line.  However, one applauded Tom for understanding how the real world deploys storage.  All good points, but I have my own theory on storage teiring...

I want to come right out and say I think that storage tiering is an incredibly smart concept.  (Now that that is off the table…) I would also say that much like the prediction that tape is ‘dead’ (I guess Data Domain didn’t get that memo), storage tiering, while it can’t be dead, because in reality, it never actually was, nor do I think it will be for a very long time.  Let’s look at the facts:

First, HSM never really went anywhere.  There is not mass adoption of HSM technology.  Second, tiering is not a technology issue.  Humans are lazy.  What do I mean?  HSM / Tiering or whatever you want to call it depends on policy.  IT can’t get any two groups in a company to decide on anything other than storage is too expensive.  When I speak to well respected people in IT the ‘real world’ (my dad), they tell me it is too difficult to get organizations to agree on when data can be archived in order to save money (and that is what this is all about really).  Finally, IT processes get in the way of a good tiering strategy.  Getting data to go one way is easy – move data to cheaper and cheaper tiers of storage until it vanishes.  Try getting it back.  That takes a lot of management tools and integration and costs just as much as doing nothing.

Remember back 8 or 10 years ago when blogs didn’t exist, and magazines did?  On the back page of one of those storage trade rags, I recall that Steve Duplessie picked Tom as the ‘Smartest Storage-Guy of 200x.’  I can’t remember which year, and Google isn’t helping, but that isn’t the point.  Tom is too smart to say something as frivolous as ‘tiering is dead’ by mistake.  He is also not a marketing guy.  So I’m placing my bets that his point is, lets help users utilize storage the way they like to consume it, simply.

The most basic value proposition for tiering is to save money inside the domain of the storage array. Tiering moves data to lower cost disk technology according to a pre-defined policy.  If your policy reflected the last time some data was accessed, tiering software would put your most active data on your highest tier of storage, perhaps SSD, and your ‘stale’ data moves to SATA.  For that luxury, you get:

a)      To fight with all the organizations within the company to decide on a policy as to when it is actually okay to move the data

b)      To spend money on a vendor’s tiering software, and pay maintenance fees, and learn how to use new software.

c)       Hope that the application doesn’t throw you a curve and want the SATA data quickly, because then you need to hurry and move it back to SSD, which would be inefficient and could be prone to error (at least historically it has).

So I think what Tom is challenging you to think about is, are you spending that money on tiering software wisely?  Vendors will tell you that it pays for itself, but does it really?  Despite the efforts of all tiered solutions to be truly autonomic, the reality is that they can’t replace a person’s decision making process, and if you could get all of your data to tier the way every organization would want then tiering would be a disruption to your process.  Additionally, I haven’t heard of a vendor offering a heterogeneous tiering solution, and not many customers buy all their storage from one vendor (as much as EMC would like this) so in the end, there really isn’t one good product available to do storage tiering so you would need many.  If this is the case, then you need people to manage all the software and tiering policies.  I thought we said this was supposed to save us money?

The hidden OPEX associated with figuring out how tiering works from each vendor in your environment will ultimately make you take pause before you deploy.  Maybe that is too much complexity to deal with for the benefit you get.

Part of the reason this discussion reared it’s ugly head has more to do with marketing than anything else.  EMC launched flash drives last year and told customers that “Capacity pricing is no longer about $/GB but $/GB/IO.”  (Of course, if you can’t sell on the rules of the game, change the rules.)  The problem is, no matter what the rules are, budgets are finite.  Selling customers on SSD (higher margin drives) meant that if users were going to buy these drives, they could only afford to put the data requiring the highest performance there so they would need to move data to a lower tier.  EMC said, “right, so we can also sell you FAST to help you with the tiering”.  The problem is, as Mark points out, it is 1.0, doesn’t work well yet and besides, for all the reasons we outlined above with regard to human nature, it really just isn’t going to take off (though I am sure the marketing group at EMC will ‘show’ otherwise).

On the other hand HDDs and Flash keep getting cheaper, so you might convince yourself that you are just fine riding that disk cost curve and working on other pressing matters, rather than deploying new tiering software.  If you take pause, maybe everyone else will as well, which means that maybe today's hype on tiering will never will be deployed widely across the industry.  Is it possible that this is what Tom was thinking?

To me, it boils down to something I’ve said many times:  new technology is easy to introduce into the data center, but new process is not.  Tiering runs the risk of disrupting process, which means buying behavior will be slow.  Plus, maybe there are other ways to reduce cost rather than using tiering.

For example, as Duplessie points out in his blog Random Thoughts for a Friday, he points out that “primary storage data reduction is going to be an in vogue conversation by the end of this year”.  So if Real-time, random access compression to primary storage can give IT what they want, significantly cheaper storage, no performance impact, maintain high availability, and be agnostic to any storage (heterogeneous) why wouldn’t they do that versus try to figure out storage tiering?  Primary storage compression takes the notion of storage tiering as a requirement and pushes it out 5 years and who knows, by then, it may work and be automated.

Now can't we all just get along and have No More Tiers?