Hello,
I recently encountered following error during Exadata GI home upgrade from 12.1.0.1 to 12.2.0.1, We encountered this error during the execution of rootupgrade.sh script on node 1 itself.
2018/06/10 02:59:18 CLSRSC-180: An error occurred while executing the command '/bin/rpm -qf /sbin/init' Died at /u01/app/12.2.0.1/grid/crs/install/s_crsutils.pm line 2372. The command '/u01/app/12.2.0.1/grid/perl/bin/perl -I/u01/app/12.2.0.1/grid/perl/lib -I/u01/app/12.2.0.1/grid/crs/install /u01/app/12.2.0.1/grid/crs/install/rootcrs.pl -upgrade' execution failed
I thought of furthur investigating this issue by running target command manually and i got following error. These errors were also logged in installation logfile.
[root@dm01dbadm01 ~]# /bin/rpm -qf /sbin/init rpmdb: Thread/process 261710/140405403039488 failed: Thread died in Berkeley DB library error: db3 error(-30974) from dbenv->failchk: DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages index using db3 - (-30974) error: cannot open Packages database in /var/lib/rpm rpmdb: Thread/process 261710/140405403039488 failed: Thread died in Berkeley DB library error: db3 error(-30974) from dbenv->failchk: DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages database in /var/lib/rpm rpmdb: Thread/process 261710/140405403039488 failed: Thread died in Berkeley DB library error: db3 error(-30974) from dbenv->failchk: DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages database in /var/lib/rpm file /sbin/init is not owned by any package You have new mail in /var/spool/mail/root
Issue was related corruption of OS level database RPM. We can validate this issue by running following command.
# /bin/rpm -qa | more
We had to fix RPM corruption issue by using following and so we can continue our Exadata upgrade.
As root OS user run the following: # rm -f /var/lib/rpm/__* # /bin/rpm --rebuilddb # echo $?
After rebuilding corrupted RPMs , using following command to validate them.
# /bin/rpm -qa | more