Verify Exadata Machine Configuration

Hello Everyone! I was recently task to perform a 360 health review and decide to share my experience with my readers. Part of Exadata 360 review, I performed detail review of Exadata configuration by verifying following 50 items. Initial Exadata Deployment usually don’t require verify following item if you have old deployment of Exadata machine, you might want to review following items at least once a year.

 

  1. Primary and standby databases should NOT reside on the same IB Fabric
  2. Use hostname and domain name in lower case
  3. Verify ILOM Power Up Configuration
  4. Verify Hardware and Firmware on Database and Storage Servers
  5. Verify InfiniBand Cable Connection Quality
  6. Verify Ethernet Cable Connection Quality
  7. Verify InfiniBand Fabric Topology (verify-topology)
  8. Verify InfiniBand switch software version is 1.3.3-2 or higher
  9. Verify InfiniBand subnet manager is running on an InfiniBand switch
  10. Verify celldisk configuration on flash memory devices
  11. Verify there are no griddisks configured on flash memory devices
  12. Verify griddisk count matches across all storage servers where a given prefix name exists
  13. Verify griddisk ASM status
  14. Verify InfiniBand is the Private Network for Oracle Clusterware Communication
  15. Verify Oracle RAC Databases use RDS Protocol over InfiniBand Network.
  16. Verify Database and ASM instances use same SPFILE
  17. Configure Storage Server alerts to be sent via email
  18. Configure NTP and Timezone on the InfiniBand switches
  19. Verify NUMA Configuration
  20. Verify Exadata Smart Flash Log is Created
  21. Verify Exadata Smart Flash Cache is Created
  22. Verify Exadata Smart Flash Cache status is “normal”
  23. Verify database server disk controllers use writeback cache
  24. Configure NTP and Timezone on the InfiniBand switches
  25. Verify that “Disk Cache Policy” is set to “Disabled”
  26. Verify Management Network Interface (eth0) is on a Separate Subnet
  27. Verify Platform Configuration and Initialization Parameters for Consolidation
  28. Verify all datafiles have “AUTOEXTEND” attribute “ON”
  29. Verify all “BIGFILE” tablespaces have non-default “MAXBYTES” values set
  30. Ensure Temporary Tablespace is correctly defined
  31. Enable auditd on database servers
  32. Verify AUD$ and FGA_LOG$ tables use Automatic Segment Space Management
  33. Use dbca templates provided for current best practices
  34. Gather system statistics in Exadata mode if needed
  35. Verify Hidden Database Initialization Parameter Usage
  36. Verify bundle patch version installed matches bundle patch version registered in database
  37. Verify service exachkcfg autostart status
  38. Verify database server file systems have “Maximum mount count” = “-1”
  39. Verify database server file systems have “Check interval” = “0”
  40. Set SQLNET.EXPIRE_TIME=10 in DB Home
  41. Verify /etc/oratab
  42. Verify all Database and Storage Servers are synchronized with the same NTP server
  43. Verify there are no failed diskgroup rebalance operations
  44. Verify the CRS_HOME is properly locked
  45. Verify db_unique_name is used in I/O Resource Management (IORM) interdatabase plans
  46. Verify Database Server Quorum Disks are used when beneficial
  47. Verify Oracle Clusterware files are placed appropriately
  48. Verify “_reconnect_to_cell_attempts=9” on database servers which access X6 storage servers
  49. Verify Flex ASM Cardinality is set to “ALL”
  50. Verify no Oracle Enterprise Linux 5 (el5) rpms exist on database servers running Oracle Linux (ol6)

 

Reference: Oracle Sun Database Machine Setup/Configuration Best Practices (Doc ID 1274318.1)