Last week I was presented with the following error on one of the Solaris 10 boxes:
svc.configd: smf(5) database integrity check of: /etc/svc/repository.db failed. The database might be damaged or a media error might have prevented it from being verified. Additional information useful to your service provider is in: /etc/svc/volatile/db_errors The system will not be able to boot until you have restored a working database. svc.startd(1M) will provide a sulogin(1M) prompt for recovery purposes. The command: /lib/svc/bin/restore_repository can be run to restore a backup version of your repository. See http://sun.com/msg/SMF-8000-MY for more information.
Having never seen this error, I was thinking: “this is gonna be interesting…”. Thankfully the error was pretty verbose so I started to disect it section by section. Yeah, service repository got hosed, somehow, and I can potentially find some usefull info in /etc/svc/volatile/db_errors. Unfortunatelly, there was nothing of use in there.
The restore_repository script mentioned gave me little more hope. I also went and checked out the page URL. After reading the page I decided to go ahead and try to restore the service repository.
I logged in to the box in single user mode and took a look at the restore script to get an idea of what it might do. Then, I ran it. Fortunatelly, the script was pretty good at doing checks and told me that I can not proceed any further because / filesystem is mounted RO. To fix this I was asked to run:
bash-3.00# /lib/svc/method/fs-root bash-3.00# /lib/svc/method/fs-usr
Once the filesystems were fixed up I ran the restore_repository script. I was asked which backup copy I wanted to restore and that was it. The system rebooted and came back up fine. This turned out to be a pretty good learning experience and http://www.sun.com/msg/SMF-8000-MY is very well worth reading.