Category Archives: Storage Companies

Storage Performance fundamentals

1. Throughput

Throughput is simply, the amount of material or items passing through a system or process. Another term for Throughput is Bandwidth. You can think of it as how fast can I transfer data in and out of my storage array.

Typically Measured as:
Data volume per second (sometimes per hour for larger volumes)

2. IOPs

Input and/or Output operations per second is the number of I/O Operations currently occurring. Some systems have fixed I/O sizes which is easier to calculate from, others will report at any given time what the I/O size is anyway.

Typically Measured as :
I/O per second = Input and/or Output / second

IOPs in a storage array is typically measured as front end IOPs to or from the host

Back end IOPs is to or from the storage media.

3. Latency ( response time )

Typically Measured as :
milliseconds or microseconds to respond to a host I/O

Storage concepts

Storage Architecture Concepts.

RAID Architectures

RAID – Redundant Array of Independent Disks

Typically drives of varying sizes can be used in a RAID configuration, however they will also size to the lowest drive format / geometry.

If you want to understand RAID concepts in more details, you can check the WIKI and various other online places rather than have me repeat it all here. Modern All Flash storage arrays that are less concerned with performance will primarily go with a dual or triple parity RAID for HA / Resilience

RAID 0

Good for performance , but no fault tolerance – striping

RAID 1

Good for read performance – mirroring

RAID 2

Forget RAID 2 – I have never in 20+ years used this

RAID 3

Forget RAID 3 – I have never in 20+ years used this

RAID 4

Forget RAID 4 – I have never in 20+ years used this

RAID 5

Good for performance. Block level striping with a single parity bit distributed over all the drives in the group. A minimum of 3 drives is required, 2 data and 1 parity but typically around 5-7 drives would be used as it allows for 1 failed drive but you don’t want to build your stripe too wide. Typically used with performance based 10K HDDs.

RAID 6

Good for resilience, similar to RAID 5, but has an additional parity bit (2). Write performance is traded off for redundancy as for every write you to also write 2 data bits so read performance could be 3x better than write performance. Often used for very wide stripes such as 18+2 or even wider when you are trying to maintain performance with data resilience.

Non RAID Architectures

JBOD

Just a Bunch Of Disks.

SPAN / BIG

Similar to RAID, but just a concatenation of drives to be used in a single volume and drives can be varying sizes, which is not good from a performance stand point.

MAID

Massive Array of Idle Drives, typically used for Write Once, Read Occasionally. Not a performance tier.

ERASURE CODING

Erasure Coding and RAID can sometimes be confused. You can use Erasure Coding in a similar way as RAID, however data is distributed and encoded (and can also contain parity data). Erasure Coding will use less capacity than RAID.

Erasure Coding is also used for things like distributed applications and Object storage providing redundancy at scale.

Move to singapore



The photo is my own. taken at East Coast Park….

It is 2019 now and I have not published anything for over a year.

Granted I don’t think I have any followers but I do hope to start publish more this year.

I have been busy uprooting my family from NZ and migrating to Singapore.

Singapore is a great central hub, I have technical responsibility for Pure Storage FlashBlade high performance file and object sales in ASEAN & Greater China.

I have spent the past almost 18 months getting my head around things and also trying to make a living ūüôā

So far things have worked out well, I have put solutions in for solutions for Oracle Data Warehouses, Big Data Analytics Platforms replacing legacy data lakes and AI/ML solutions integrating NVIDIA DGX-1 AI appliances.

In Pure Storage context, these AI solutions are called an AIRI, AI Ready Infrastructure .

Business Lessons From a Boat Race

Earlier this week Emirates Team New Zealand (ETNZ) took the Americas Cup from Oracle Team USA (OTUSA) and it was a great day to be a Kiwi.

There are some key lessons to be learnt from this on why ETNZ won. Although to some this was just a boat race, it can also be viewed as a modern day version of David and Goliath. ETNZ as David and OTUSA as the well funded, well provisioned giant Goliath. With two boats that for the layman look the same but are not.

ETNZ knew two things coming into this race, you can’t rely on winning if you do things the same way as everyone else or as they have always been done and you need a world class team of support and specialists around you to ensure everything goes off without a hitch.

In traditional yachting you rely on ropes and pulleys and hands on man-power to trim the sails and guide the boat. ETNZ used technology and innovation to do things smarter.

They asked themselves how could we do better? They took what looked like a regular sail boat and reinvented it. They hired a world class design team from non traditional sailing backgrounds, a surfer from the U.S. that could build a drone to show them every aspect of the boat from every possible angle in a way that had never been done before.

ETNZ changed the “engine” of the boat. In the latest Americas Cup boats they rely heavily on hydraulic power, traditionally generated by human grinders that peddle with their arms to build hydraulic pressure. ETNZ looked at this process and decided it was inefficient so they swapped traditional grinders for Olympic cyclist. Cyclists obviously use leg muscles, a much bigger muscle group to do the same job. The Cyclists could also stay lower in the boat, creating less windage. The ETNZ cyclists were specialists in their on board jobs and did not have to worry about doing other tasks.

This innovation allowed ETNZ to build hydraulic power much faster and more efficiently than normal, requiring much less energy expended.

The net result was that the ETNZ boat was more efficient because of the technology and innovation that went into it and using the most appropriate tool for the job.

OTUSA are a very smart team, they knew that the Kiwis had a technical advantage, so they tried to imitate them. They tried to copy the technology but in the end their boat was not built to take advantage of the innovation and their retrofit was a poor imitation and in the end, they took it off.

The last point I want to make is that if you capsize as ETNZ did and if your sailing hard you probably will, you want to make damn sure that you have a the team behind you that is there to support you is world class and that they can get you back on the course quickly and efficiently so that you can continue your race.

What lessons can be learned from this?

For the layman, many things that are similar look the same. The devil is in the detail, was this device built for the technology it is using or was the technology added as an afterthought because someone else has done it.

Something that is purpose built for a task is much better than something that is bolted on or retrofitted later. Pure Storage is the founder of the modern All Flash Array. The FlashArray was purpose built to take advantage of flash in the modern enterprise.

Technology is evolving to make things more efficient and give you a tactical and operational advantage, use it. Look to companies that are innovating, that want to do things better to give you a business advantage.

Build solutions around world class teams that can challenge the norm and be innovative. Pure Storage has a SatMetrix certified NPS of 83.5, that’s huge, that’s world class and it has been independently verified. Customers that use Pure, love Pure and are happy to tell you and anyone that will listen about it.

My original article
https://www.linkedin.com/pulse/business-lessons-from-boat-race-alex-johnson

The true NDU

I remember the first time I saw colour TV. I came home from school and sat down to watch my favourite after school show. 10 minutes in and I was asking my parents what was wrong with the TV because it kept changing channel. My father laughed at me.

On closer inspection, I noticed that not only was the massive intricately carved TV box that we bought back from Singapore gone, it was replaced with a smaller simple brown box, that was displaying colour and had my father had a wireless remote in his hand! Wired remotes were still common then.

My world had changed forever. Technology had evolved to enhance my user experience and I loved it.

Over the course of the next 30 or so years I was lucky enough to have experienced several such moments. Enhancements in technology have allowed us to transform the user experience in many areas. How amazing was it when you didn’t have to tie up the telephone line any more to be online!

10 years ago when we wanted to lifecycle storage, we we doing so with storage virtualization and that was awesome, but you run into the problem of how you lifecycle the lifecycle appliance? Then came storage migration at the hypervisor layer which was great for virtualised systems, but not so good for physical.

Last week I had a wow that’s so cool moment, when during the business day we non-disruptively upgraded a customer from our legacy hardware into our new hardware.

You might be saying to yourself ‚Äď so what, I do that all the time! or everyone can do that, its easy!

I beg to differ. In recent times this has become easier for folks running virtual workloads and for me as a storage guy VMware changed the game by allowing people to non-disruptively move data stores between storage devices. If you had a modern array, the array would also be considerate enough to do most of the heavy lifting for you as VMware created the VAAI APIs that allowed the storage to offload a bunch of the t10 SCSI commands, thus freeing up compute CPU cycles. Old news I know, but there are still legacy storage arrays that don’t support these things.

What I want to be clear on though was that I didn’t have to use any software tools to do this!

The architecture of the Pure Storage array is what allows me to do it. More importantly it allows me to do it without impacting host IO.

The company I was doing the upgrade for is a very sensitive company, with sensitive data and conservative customers. We started the upgrade at 10:30 am and it was complete in a few short hours.

Not only did we swap the storage controllers, but we upgraded the capacity at the same time.

The measure of success was no help desk calls and we were successful.
To be clear, this was done during business hours with a full complement of staff working and apps running.

I love my job. I work for a disruptive company changing the industry of technical storage.

Maximum HBA port queue depth in an all Flash Array

Per port queue depths is a common question we hear and how we compare to these limits other vendors publish based on array model. 

Pure does¬†not have¬†queue¬†depths assigned on a per port basis and we do not modify the Qlogic HBA per port¬†queue¬†depth¬†of 4096.¬† We also do not issue SCSI “busy” responses back to hosts (instead, we will¬†queue¬†all I/O in the array) there is occasionally some host¬†queue¬†depth¬†adjustments needed for some environments.¬† The goal is to not unnecessarily limit outstanding I/O’s to the array during normal steady state, but also to avoid a ¬†situation in cases of massive bursts of I/O from many hosts concurrently.

With legacy mechanical disk arrays, it was important to have as many outstanding I/O’s as possible due to the seeking and rotating going on. ¬†

With all flash arrays, and Pure in particular, it is not necessary to keep the¬†queues¬†full and the I/O responses are returned much faster as well, meaning it’s pretty hard to keep the¬†queues¬†full (at least for most organic workloads).¬† One of the key values for Pure is the simplicity – you should not have to do the same types of computations and planning like you are used to with mechanical disk storage.¬† In some cases, there may be some adjustments in aggregate for all hosts connected to the same array to mitigate a IO storm¬†(ensuring host-level fairness), in most cases it is not necessary as we remove the need to do per-port planning.

Auckland vMUG July 2015

We had a great time last night (30-07-15) at the @PureStorage sponsored  @AucklandVMUG event. Approx 65 attendees registered and we would have had at leas that number in the room.

The photo below

BlueStone Room
BlueStone Room

Lots of great guests;

  • Nick Bowie kicked us off even with getting over a cold ¬†(@NickBowieNZ). VCDX5-DCV #202, VCP6-NV, NPP #43.
  • Craig Waters took us through Flash in the Data Centre ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬†(@cswaters1). ¬†VCAP-DCA/DCD, VCP-DV 3-5, vExpert 2012-14
  • Alastair Cooke spoke initially about training then gave us a good talk about your career and how to manage it yourself ¬†(@DemitasseNZ) vExpert

Nick Bowie
Nick Bowie

Lots of great attendance from the members of the Auckland chapter of vMUG, plus a couple of sneakie entries from other vendors wanting to check us out.

We had a major prize of a 250GB 3D NAND SSD that was won by Diego Nievas from Datacom after a draw by Stuart Blythe, Country Manager of Pure Storage in NZ

Diego Nievas wins SSD
Diego Nievas wins SSD

Kudos to Sanit, Michael, Steven and Ping for putting on a great event.

Stop the negative

Do you follow Twitter, LinkedIn or some other social media service to stay informed? Are you sick of the vendor bashing?

Like many of you, I have been doing what I do now for many years.
Similarly, like most of you – I need to keep up with what is happening in my chosen profession. For me that is the world of technical storage.

I am a storage and data management professional, so I need to keep up with all of my competitors information, and any new products and services on the market that relate to what I do.

So I would assume most people reading this would do a similar thing – given the amount of people that refer to LinkedIn or Twitter, I would say thats an accurate assumption.

Let me ask you a couple of questions…….
What would you do if you went looking for a new house and saw the estate agents sign out front, with another agents sign tacked underneath saying that the first agent had just made up everything and to so go talk to them?
What would you think?

How about if wanted a new car and in the yard that you are looking another car yards sign was stuck to the window of the car you were interested in saying that the first car yard doesn’t know what they are talking about and its all lies – come see me and I will sell you a similar car?
What would you think?

That is what is happening all over social media these days and it is making us as vendors look like children.

I admit to a bit of social engineering on some occasions.
Even recently, a friend of mine that works for a competitor posted an article from a third party research firm. I posted a comment on his post that referenced my company talking to the author of that article. Thats fair and reasonable, right?
At no point did I detract from his comments, I even agreed with them in a later comment – but I injected a positive article from my company on his post so his followers would see and hopefully read it.

Thats fair right?

What I think is not fair is when other vendors go off on a rant claiming your information is false, misleading or wrong. Its the equivalent of a child’s tantrum.
I could share countless articles like this, but I wont.
Some people would probably comment on this saying it was the company I now work for, going hard against its biggest competitor that started this landslide of negativity. They might be right as I am not sure when it got as bad as it has.
What I do know is that where I work from the top down – they are saying don’t go negative on social media. I understand other vendors are doing the same.

I just don’t believe everyone else is subscribing to the same ideals.

In my world we all have competitive intelligence, but the reality is this is often outdated or inaccurate so needs to be taken with a grain of salt.
In a previous role, I was part of the competitive community, so I know how difficult it is to keep up and maintain accurate competitive information – and that most of it is in the interpretation of the writer.

Going negative on social media is bad and it needs to stop!
If you don’t have anything good to say, then don’t say it.
In the context of what we are talking about, that would be doing as I did above, reference an article of your own if its positive, or maybe highlighting something that your product does better without saying that the competitive product is bad.

Write your own article that accentuates your product over your perceived weakness of the competitions – educate!
Be passionate, market yourself or your company better.

At the end of the day, people move, change jobs, but companies stick around and some folks have long memories so stay classy and promote you, and your own company.

My suggestion, if you see this happening – call it out, let them know what you think. If we don’t stop it then people will simply stop educating themselves and we all lose!

Stop the negative!

negative-thumbs