Is there a compelling reason to virtualize Exadata Machine?


Virtualizing Exadata machine has become an important deployment decision for many Exadata customers and most of them like to explore or at least discuss virtualization to see if there is any benefit for them. I believe you should have a good use case to virtualize Exadata machine and it should not be your standard install. Keeping that in mind I like to list following use cases where it makes sense to virtualize Exadata machine.

Cost Saving: With the introduction for Elastic configuration and Capacity on Demand (COD), you can already save significant amount of money on licensing and initial investment. With Exadata Elastic configuration option, you can build Exadata with almost any configuration of compute and storage servers. And Capacity on Demand (COD) option allows you but Oracle license in increment. With minimum of 40% must be license, you will be able buy 1/8th rack by only licensing 8 cores per server. So how will OVM with safe money on Licensing? Additional Cost Option Licensing. Virtual machines on Exadata are considered Trusted Partitions and therefore software can be licensed at the virtual machine level instead of the physical processor level. Without Trusted Partitions, database options and other Oracle software must be licensed at a server or cluster level even though all databases running on that server or cluster may not require a particular option. Even with Unlimited License Agreement (ULA) organizations don’t have unlimited licensing for everything (Golden Gate, Advance Security, Advance compression, etc). Some of the licensing options are very expensive and can end up playing key role in your decision to buy Exadata machine.

Compliance: Secondly, I see compliance as another reason to virtualize Exadata Machine. There are different types of compliance requirements HIPPA, PCI DSS and Certifications. We already have a clear definition of HIPPA, PCI DSS compliance requirements and none of them will require you to virtualize Exadata machine. But certification is different, different software and hardware vendors will have set of Software & Hardware requirements to certify their application.  You might be required to isolate your workload at database level or cluster level or operating system level. For Example, if your databases contain sensitive client data from different business partners you might be required to isolate data at operating system level or even physical level. You can achieve different level of isolation with Exadata Machine without using OVM. You can have additional Oracle RDBMS Homes to provide Oracle binary files isolation, you can also have different disk groups to provide storage isolation and it is also possible to have a separate physical cluster if you have half or full Exadata rack. But you won’t be able to have two separate physical Oracle Clusters on quarter or eight rack.  Using VM’s you will be able to install two or more VM Oracle clusters and achieve operating system level isolation.

Consolidation: Exadata is optimized for both OLAP and OLTP database workloads. Its balanced database server and storage grid infrastructure also makes it an ideal platform for database consolidation. Consolidated environments running on Exadata can use Oracle Virtual Machine (OVM) to deliver a high degree of isolation between workloads. This is a very desirable feature for hosted, shared, service provider, and test/dev environments. Using OVM, multiple software clusters can be deployed on the same Exadata Database Machine, which enables consolidation of applications that have specific clusterware/rdbms/maintenance needs. Not every organization have a separate Exadata machine for development and performance testing. Ideally you should have development and test environments on Exadata machine, so you can take full advantage of Exadata features like smart scan and offloading. You would also like to separate prod, pre-prod and test environments to define separate maintenance windows. For example, if mission critical applications sharing the same Exadata machine with development or test systems, then the frequent changes made in development and test systems will impact the availability and stability of the mission critical applications.


Don’t do it unless you have good use case for it.

Virtualized Exadata Machine (Isolation vs Efficiency)

Virtualizing Exadata machine has become an important deployment decision for many Exadata customers and most of them like to explore or at least discuss virtualization to see if there is any benefit for them. Since I have already been part of those conversations, I decided to share my thoughts on this topic to help my readers.

Oracle started supporting Exadata virtualization while ago and it’s free. You might want to virtualize your Exadata Machine for many reasons (Consolidation, Security, and Compliance) and end result will be to achieve some level of Isolation. Isolation is probably one of the main reason to virtualized Exadata machine. And if you are planning to virtualize your Exadata, keep in mind that everything (CPU, memory, disk) will be hard partitioned. Even though you can over-provision CPU’s, Oracle recommend strongly against over provisioning any resources. With dedicated CPU’s, memory and disks you will be able to achieve great isolation but it will not be an efficient use of Exadata machine resources. For instance, virtualization will provide you opportunity to have different patching cycles for each Exadata VM cluster but not without maintenance overhead. I have worked with Exadata Rack with up to 3 VM’s and it was not fun patching them, imagine if you have multiple virtualize Exadata machines. Remember Oracle releases around 4 bundle patches a year and you need to apply at least 2 of them to be in compliance for Oracle Platinum Service. Additionally, since everything is hard partitioned in virtualized Exadata machine, you will not be able to use IDLE hardware resources from Other VM’s. Hence you are wasting very expensive Hardware and Software resources.

It’s also important to understand that there are many levels of isolation like physical level, OS level, Storage Level, cluster level or RDBMS level and you can still achieve some level of isolation without virtualizing Exadata machine. For example you can have multiple RDBMS homes, different ASM disk groups and isolated network using VLAN.  I am not against virtualizing Exadata Machine but you should have very good use case for it. I would suggest combining above mentioned isolation strategies with 12c multitenant option to achieve excellent efficiency. But again if you are required to isolate everything at OS level, virtualizing Exadata Machine using OVM is your only option. Even though Exadata VM’s are also great for consolidation but best strategy would be to combine VM’s with database native consolidation options like multitenant. Exadata VMs have good Isolation but have poor efficiency and higher maintenance. Virtualizing Exadata Machine should not be your standard built, you should always consider bear metal install over Virtualized Exadata install.

Virtualizing Exadata Machine Using OVM

It’s been a while since Oracle started supporting OVM with Exadata Machine. It means you can virtualize your Exadata Machine using Oracle VM technology with no additional cost. Ideally, Oracle VM with Exadata machine should be use for database consolidation or isolation only but you are not restricted. You can use Exadata VMs to install other products like management or ETL tools but it’s not recommend to run major applications like SAP or EBS. Currently Exadata VM are only certified to Oracle enterprise Linux, may be in future Oracle will start supporting other operating systems. Please see below diagram to compare quarter rack Virtualized Exadata Machine with Bare Metal.

Virtualized Exadata 01

Virtualized Exadata machine architecture can be significantly different than the Bare Metal one. With Bare Metal install, you have one Oracle cluster for whole Exadata machine, unless you physically partition your Exadata machine. With Virtualized Exadata Machine you have 1 management domain (domO) and at least 1 user domain (domU) on each node, depending on the number of VM clusters being deployed. Management Domain (domO) is automatically created with 4 vCPU, 8 GB RAM, with no Oracle GRID and RDBMS software installed in it. Each user domain (domU) should be sized carefully based on the databases you are planning to host in future. With virtualized Exadata install, you only virtualize compute nodes and share storage between them. This also mean that all the VM clusters will have their own dedicated ASM disks groups and physical disks.