Databases and Storage - The New Frontier

Databases Storage – The New Frontier


As database and storage administrators and engineers, it is important to understand the effect that storage has on database performance.

It is important to understand that databases are an extension of your storage and requires a holistic view when designing and managing solutions that use the database.

This article gives an overview of the Oracle database storage landscape – past, present and future. It’s not meant as an in-depth technical treatment of the topic but opens the door for more discussion and engagement about database storage.

Evolution

Both storage and databases continue to evolve (albeit at different paces). About a decade ago the enterprise storage of choice was the Storage Area Network or SAN array. At about the same time the current release of Oracle was 10gR1, which introduced Automatic Storage Management (ASM), which provided mirroring and striping capabilities for data files. Database sizes were typically in the hundreds of gigabytes. With the current database release of Oracle 12c, new storage features such as Automatic Data Optimization (ADO) and Heat Maps have been introduced to enable automatic and intelligent storage tiering within the database.

Data growth has also exploded and sizes ranging from tens to hundreds of terabytes are now common. Features such as advanced compression (with hybrid columnar compression with Oracle ZFS appliance, Pillar Axiom orand Exadata) can significantly reduce the amount of storage required. Additionally when it comes to data resilience, the landscape has also significantly changed from the use of magnetic tape devices such as LTO to highly available disk-based solutions with de-duplication and replication. We’ve also witnessed the proliferation of server virtualization for improving hardware efficiency. Given the evolution rate of these technologies it becomes increasingly important for storage management professionals to keep up and arm themselves with the knowledge needed to work effectively.

Performance

“Number rules the universe.”
-Pythagoras
When it comes to storage performance, we’re often directed to bombarded by three specs – latency, bandwidth and IOPS. But does this tell the whole story? The short answer is “No”. Each application has its own I/O requirements that are not captured by vendor benchmarks. Data access workloads (random versus sequential) and ratios, I/O sizes all play a part in storage performance. Your data warehouse on fibre-attached SAN arrays with rotating disks may give excellent performance (throughput and bandwidth) but your OLTP workload for an online application on the same array may totally suck! Can you say latency?

How can one measure storage performance? For seasoned Oracle database professionals the answer may seem obvious – response times. But which metrics are relevant for measurement when storage is involved? The Oracle database has a well-instrumented wait interface. However, it is not perfect due to the nature of sampling. That said tools such as Automatic Workload Repository (AWR) and Statspack can provide a wealth of information about database performance and in particular I/O metrics (including histograms). From a back-end storage perspective, there are tools from storage vendors that provide relevant metrics. Some vendors even provide plug-ins for Oracle Enterprise Manager Cloud Control 12c to enable central collection and reporting of storage metrics.

Speaking of measurement – you can find great tools to benchmark your Oracle database storage such as SLOB and fio.

Flash

Up until a few years ago rotating disks had all but dominated enterprise storage. While not a brand new technology, flash storage has started to proliferate the IT infrastructure in the form of SSD and more recently all-flash arrays. Flash provides low-latency I/O. Traditional hard-disks have service times in the milliseconds while that of flash is typically less than 1 millisecond. Does that mean that you should dump all your disks for a flash based solution? While flash is great at providing random I/O, spinning disk is still OK for sequential I/O. While moving all your redo logs to Flash may improve response times, it may not give you the best bang for your buck, especially if there are other I/O bottlenecks. With that said there are other considerations such as operational cost that goes into choosing a flash array. There are other issues such as durability and resilience that affect fFlash technology but these are becoming non-issues as the technology improves.

Software is the new hardware

So what’s next for database storage? Flash technology is now becoming mainstream and the industry is looking towards [hyper] converged infrastructure. Ah yes another marketing buzzword. In a nutshell, converged infrastructure involves putting storage, network and computing resources in a single hardware unit. It’s no surprise that storage has followed suit in the form of software-defined-storage (SDS) and storage virtual machines (SVM). As an example, some vendors virtualize storage controllers in software. This allows for an agnostic storage back-end i.e. DAS, SAN/FC, NFS. Interestingly, Oracle has continued to implement more storage related features at the database layer e.g. Flex ASM, ACFS aka Cloud Filesystem – a POSIX compliant general purpose filesystem that includes Snapshots, Replication along with DBFS and Direct NFS.

By virtualizing hardware storage resources businesses can become more agile (yes another buzzword) and efficient.

What’s next?

With the new storage frontier upon us, how does one keep up and learn more about their database storage?

The IOUG Storage SIG is a great way to keep on what’s going on in the world of Oracle and database storage. There are whitepapers and webinars from Oracle product managers, partners and end-users. Also check out the references section for links to whitepapers, articles and blogs from notable experts in the Oracle community.

References

Kevin Closson (SLOB author) - http://kevinclosson.wordpress.com/slob/

Flashdba has an excellent series “Storage for DBAs” http://flashdba.com/storage-for-dbas/
James Morle’s book – Scaling Oracle 8i (still relevant) and SANE SAN 2010 whitepaper- http://www.scaleabilities.co.uk/scalingoracle8i/
Frits Hoogland - http://fritshoogland.wordpress.com/
Michael Webster - http://longwhiteclouds.com/oracle/
Kyle Hailey Lies, Damned Lies, and I/O Statistics– http://www.oraclerealworld.com/lies-damned-lies-and-io-statistics/

Comments

Post a Comment

Popular posts from this blog

Setup a Wordpress site in 10 Minutes using Oracle Container Cloud Service

Image
Oracle Container Cloud Service Oracle Container Cloud Service lets you easily deploy applications based on docker images. If you don't have an existing application you can create one on docker. However, the easiest method to get a full application up and running is by deploying a stack. Oracle Container Cloud Service provides several example stacks that you can deploy, including web servers based on NGINX or Apache, MongoDB database, and Wordpress. You can find more examples on Oracle GitHub Site . Let's see just how easy it is to spin up a Wordpress site using the Oracle Container Cloud Service. Prerequisites: Access to Oracle Cloud Account Subscription to Oracle IaaS Service First, go to the Oracle Container Cloud Service console and launch a new service. In this example, I created a service with three containers. One manager and two worker nodes. Choose an appropriate "shape" for your compute instances. Note: Oracle uses the term "shape" to r...
Read more

Oracle Home Cloning: Tools and Techniques

One of the many necessary and frequent tasks that a database administrator performs before creating a database is installing the Oracle database binaries. These binaries form what is known as an Oracle Home (referenced by the ORACLE_HOME environment variable on Linux and UNIX environments). Unfortunately, this is also one of the most mundane and uninteresting tasks a DBA can perform. So like all good DBAs know, the best way to deal with such repetitive task is to automate it. Automating installations of Oracle database binaries enables us to reliably repeat the process to enable faster installation as well as provide identical copies of the software in multiple environments. Fortunately, Oracle allows us to clone the Oracle database binaries from one environment to another. Cloning your Oracle binaries from a source copy with all the necessary patches applied enables us to have gold images which can then be deployed to multiple environments. The procedure for cloning an Oracle home...
Read more

Using XFS on Oracle Linux

Install xfs packages on Oracle Linux 6.6 via Oracle Public Yum [root@matthew ~]# yum install xfsprogs Loaded plugins: refresh-packagekit Setting up Install Process public_ol6_UEKR3_latest | 1.2 kB 00:00 public_ol6_UEK_latest | 1.2 kB 00:00 public_ol6_UEK_latest/primary | 19 MB 00:02 public_ol6_UEK_latest 407/407 public_ol6_addons | 1.2 kB 00:00 public_ol6_addons/primary | 77 kB 00:00 public_ol6_addons 269/269 public_ol6_latest | 1.4 kB 00:00 public_ol6_latest/primary | 46 MB 00:05 public_ol6_latest 29302/29302 http://public-yum.oracle.com/beta/repo/OracleLinux/OL6/uek3/x86_64/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found" Trying other mirror. Resolving Dependencies --> Running transaction check ---> Package xfsprogs.x86_64 0:3.1.11-1.0.5.el6 will be installed --> Finished Dependency Resolution Dependencies Resolved ===================================================================================================...
Read more